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ABSTRACT 

The purpose of this investigation was to assess the instructional 
value of advance organizers, instructional objectives and adjunct 
questions in preparing instructional prose. Previous research relating 
to each of these manipulations led to some questions both about their 
forms and the methodologies most commonly used. Alternative forms of 
organizers, objectives and questions were derived from analysis of their 
hypothesized functions and their current use in applied instruction. 

The two forms of each manipulation, one drawn from current use in the 
literature and one devised in the light of instructional practices, were 
investigated within a factorial research design. 

Data analyses indicated that the prose organizer performed signi- 
ficantly better than the graphic-word organizer. This unexpected effect 
was accounted for in terms of students' possible preferences for and 
abilities with verbal rather than visual information sources. Failure to 
obtain differences between either form of objectives and questions was 
attributed to their lack of use by the students. This lack of use was 
thought to result from relatively low motivational states inherent in the 
research context. 

Differences obtained between scores of students given instructional 
manipulations and those vei text only were discussed as supporting 
the contention that equal available time does not lead to equal perform- 
ance. However, low magnitude of scores within treatment groups and low 
mangnitude of differences in treatment effects suggest that further 


research demonstrating stronger facilitation effects are necessary before 
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firm conclusions, relative to the value of inserting organizers, 
objectives and questions in text, can be made. 

Differences between prose and graphic-word organizers and changes 
in research strategies were discussed as areas which should receive 
further research attention. It was concluded that students' motivation 
levels may be a more critical issue in relation to "instructional" 
research in which an attempt is being made to promote learning phenomena 
than is the case in "learning'' research where an attempt is made to 


study and draw inferences about the learning process. 
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CHAPTER I 


The purpose of this investigation was to assess the instructional 
utility of advance organizers, instructional objectives and adjunct 
questions as manipulations which promote learning from prose materials. 
Each of the manipulations was examined in two forms, a form which is 
commonly used in the published literature and a new form derived from 
instructional practice. The relative utility of both forms of the three 
manipulations in improving learning from prose was assessed by means of 
a research design in which one or the other form of all three manipul- 
ations was presented within prose materials. 

In Chapter I, the research rationale and literature reviews relating 
to organizers, objectives and questions are presented. Problems 
identified by the literature reviews are discussed in terms of some 
current instructional positions and procedures. New forms of organizers, 
objectives and questions are derived from these discussions. 

In Chapter II, the production of materials, the research design and 
the methodology is described. The data and results are presented and 
discussed in Chapter III and Chapter IV presents some conclusions, 
implications and suggestions for further research. 

The basic orientation of this research is instructional which, 
according to Gagné (1974), means that attention is directed towards: 


... the set of planned external events which influence the process 
of learning and thus promote learning. ... The external events do 
not produce the learning -- instead they potentially support 


processes which are occurring within the learner. (p.5) 


While instruction is not media specific, consideration of 


instructional issues must be made manageable by constraining or limiting 
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the domain of interest. In this investigation, prose or text materials 
will be the particular medium of interest. This selection is based in 
part upon Gagné's statement that: "The kind of learning the reading of 
books accomplishes most is the learning of information (1974, p.6)" and 
in part, upon the recognition that prose materials remain one of the most 
readily available sources of information. 

While prose materials are used extensively in many learning contexts, 
relatively little attention has been directed towards determining what 
"planned external events' are necessary to convert ordinary text from 
which students might learn into instructional prose which will support the 
students' learning processes. No systemmatic body of research or theory 
exists from which prescriptions for the production of prose instruction 
may be derived. There are, however, a number of sources of information 
which can contribute to this process. 

Glaser's basic teaching model, for example, (in DeCecco & Crawford, 
1974) provides a graphic display which can be used to organize the 


variables which will be under consideration. 


Instructional Entering Instructional Performance 


Objectives bang! Behavior gad Procedures ea Assessment 


The four blocks represent four major components of the instructional 


sequence and are equally applicable to classroom or prose instruction. 
According to this model, effective instruction must provide for the 
requirements of each of these interrelated components, the instructional 
objectives specifying the information or skill to be acquired, the 
entry behaviors relating to the students’ existing information, Ski, 


ability and motivation, the instructional procedures which will draw the 
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students from their initial information or skill position to the one 
specified by the objectives, and performance assessment which will 
ascertain whether or not the desired change in behavior has occurred. 

In order to use prose materials as essentially SuRONoHSis instruc-— 
tion which is intended to promote learning without the assistance of a 
teacher, it seems important to examine some of the ways in which prose 
might be modified to provide support for students learning processes. 

Three manipulations have been selected for use in this investiga- 
tion for several reasons. First, while all three manipulations arise 
from or are associated with somewhat different areas of research and 
theory, organizers from Ausubel's (1963) theory of meaningful verbal 
learning, objectives from military and industrial contexts via Mager 
(1962), and adjunct questions associated ay Rothkopf (e.g. 1970), 
Frase (e.g. 1970) and Anderson (e.g. 1970), each of these manipulations 
is viewed by their proponents and have received some empirical support, as 
"planned external events" which can support the students learning process. 

The second reason for the selection of advance organizers, instruc- 
tional objectives and adjunct questions is their particular appropriate- 
ness for use with prose since they are themselves most frequently used 
in printed form. A third reason for the selection of these manipulations 
is their discreteness from the prose material itself. This discreteness 
means that the manipulations can be changed without directly changing the 
prose. Thus, the author of the organizers, objectives and questions need 
not necessarily be the author of the prose. This is a particular advant- 
age when it is remembered that the individual preparing instructional 
prose may not be an expert in the subject matter of the prose. 


A fourth, and final reason for the selection of organizers, 
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objectives and questions in this investigation is the fact that these 
manipulations are used in applied instructional contexts (e.g. Course- 
ware, Inc., 1974) and the view expressed by one of these practitioners 
(Merrill, 1976 personal communication) that if it is not possible to do 
anything else, the provision of organizers, objectives and questions may 
be a sufficient condition for promoting learning from prose. 

While the rationale underlying the use of organizers, objectives 
and questions is relatively strong and cogent, it is also true that each 
of these variables is associated with some problems of a theoretical or 
applied nature. The literature reviews which follow describe the pro- 
blems associated with each of the manipulations. Guidance is then drawn 
from current instructional practices to produce revisions to the mani- 
pulations which may improve them for use as "planned external events" 
which will support students' learning processes. 

A Selected Review of the Literature 

All three of the manipulations to be considered have received 
substantial attention in the published literature for at least 10 years. 
The result has been a very large collection of reports and position 
papers. In addition, the literature relating to all three manipulations 
has been reviewed within the past few years and these reviews are 
available in published form. This review will therefore refer directly 
to the published reviews and discuss in detail only those papers which 
have some particular relevance to this research. 

Advance Organizers 

According to Ausubel (1968) the single most important prerequisite 

for meaningful verbal learning is the existence, within the learner's 


cognitive structure, of suitable “anchoring ideas" for the new learning. 
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He views the process of learning as the organization of new material in a 
hierarchical manner within the learner's cognitive structure. Thus, to 
Ausubel, support of the learner's processing requires provision of 
appropriate anchoring ideas which can be made available by means of 
"advance organizers". 

Advance organizers are materials prepared at a "higher order of 
generality and inclusiveness" than the new learning materials and their 
relevance to the new learning materials is made explicit. 

In a recent review by Barnes and Clawson (1975) 32 studies are 
described. Of the studies presented only 12(approximately 38%) showed 
significant facilitative effects for the use of advance organizers in 
learning from prose. In addition, Barnes and Clawson (1975) analysed 
the studies in terms of length of study, ability level of students, type 
of organizer, and cognitive level of the learning task and failed to 
find clear patterns emerging. 

It is interesting to note that a number of other researchers, 
(Dooling & Lachman, 1971; Bransford & Johnson, 1972; Dooling & Mullet, 
1973) whose papers were not included in the 1975 review, have suggested 
that advance organizers may have a facilitative effect on performance 
scores but only when the new learning material itself is not conducive 
to the learner's production of his own organizer. For example, Dooling 
and Mullet (1973) presented vague metaphorical stories with or without 
an organizing title. They found, with material from which a student was 
unlikely to draw his own organizing title, the presence of a title 
improved performance. It is important also to note that these authors, 
while referring to their manipulations as advance organizers, actually 


employed "organizing titles" and "single word themes". These uses of 
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6 
"advance organizers" seem to differ in terms of their length and complex- 
ity from the advance organizers used by Ausubel. 

Another type of organizer found in the literature may have 
instructional potential. Barron and Stone (1974), for example, presented 
learners with a "graphic-word" organizer. This organizer appears to be 
essentially a flow-chart of the main concepts in the learning materials 
and of the relationships among concepts. In their research, one group 
received the experimenter-produced organizer prior to the new learning 
material and was encouraged to discuss and elaborate the organizer with 
the experimenter. A second group received the learning passage first 
and was then encouraged to generate their own organizer in small groups. 
A third group was simply asked to read the material. In terms of 
performance scores, the post-organizer group exceeded the pre-organizer 
group which exceed the control group. Barron and Stone (1974) argued 
from these results that organizers may have their influence on students' 
"reading/learning process rather than ... on cognitive structure." They 
do not, however, offer much explanation for the superior performance of 
the post- compared to the pre-organizer group, nor do they discuss the 
issue of interaction with experimenters in the production of pre-organ- 
izers or interaction in small groups in the production of post-organizers. 
The issue of the relative efficacy of graphic-word organizers in 
comparison to prose organizers, therefore, seems to have arisen but in 
this research, has been confounded with interaction between student and 
experimenter production. 

What an organizer is and how it works, if indeed it does work at 
all, are issues towards which some comment has been addressed. Barnes 


and Clawson (1975), for example, conclude that research using advance 
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organizers has suffered from the absence of an operational definition of 
the term. This concern has also been reflected by Blanton (1971), 
Barron and Cooper (1972), and Hartley and Davies (1976) all of whom 
criticize the lack of operationalization of advance organizers. 

Summary and Critique 

In. reviewing organizers, Barnes and Clawson (1975) conclude that 
there is no substantial evidence supporting advance organizers as 
facilitators in learning from prose. There seems, however, to be some 
indication in the research cited by these reviewers that the distinction 
made by Ausubel (1968) between derivative and correlative subsumption may 
have been ignored. According to Ausubel (1968) derivative subsumption 
occurs when the learning material is a specific example of a concept 
already existing in cognitive structure. Correlative subsumption occurs 
when the learning material is an extension or elaboration of previously 
learned propositions (Ausubel, 1968, p. 100). Because he views the con- 
sequences of obliterative subsumption to be more severe in the case where 
correlative subsumption is the desired process it seems that this is also 
the situation in which advance organizers are most important and effective. 
With relatively little detailed information available from which to 
establish whether derivative or correlative subsumption was the process 
being promoted it is not really possible to comment further. However, it 
is possible that research in which the learning material was a specific 
example of a concept already existing in cognitive structure was among 
those which failed to demonstrate the usefulness of advance organizers. 

Barnes and Clawson (1975) make a number of suggestions for 
additional research using advance organizers. Specifically they suggest: 


Studies using a wide variety of nonwritten advance organizers 
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that are operationally defined and constructed according to 
the general criteria espoused by Ausubel should be conducted. 
(p. 656) 


It is this particular suggestion which will be considered in terms 
of some current instructional practices. 

An Instructional View of Advance Organizers 

In the literature reviews cited, the only consensus seems to be an 
awareness that while organizers sometimes appear to facilitate learning 
from prose, their use is associated with methodological difficulties 
particularly in terms of an operational definition. 

There are two characteristics of Ausubel's (1962) version of the 
organizer which are not made explicit from the definition. First, it is 
450 words long; second, the organizer uses a number of relatively 
sophisticated technical terms. These characteristics of length and 
language seem to be potential problems when an organizer is intended for 
a student's use. This may account for the comment from Barron and 
Cooper's (1972) students that they viewed the organizer as "additional 
learning materials." It may also be the reason why Ausubel's use of 
organizers has been most successful when the organizer is presented on 
several occasions prior to the new learning materials. 

However, instructional psychologists, for example, Merrill (personal 
communication, 1976) also advocate the use of some type of "organizer" 
or introductory material which is similar to Ausubel's advance organizer, 
in intent if not form. Ina list of Basic Requirements (Courseware, 
Inc., 1974) a description of "instructional sequence relationships" is 
advocated. In essence, this description is usually a relatively short, 
often schematic, flow chart of the major concepts of an instructional 


sequence showing how each concept is related to other concepts. The 
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intent of this "display" seems basically to show the student where he 
has been, where he is and where he is going with respect to the learning 
material. Merrill (personal communication, 1976) considers this 
"sequence relationships display" to be comparable to Ausubel's advance 
organizer since its intent is to provide the student with aes ei aes in 
which he might relate the new learning to what he already knows. It is 
not clear whether this ''display'"' is intended to be a superordinate 
organizer or an overview. 

There are two characteristics which make this instructional organizer 
different from Ausubel's organizer. First, te 22 relatively short in 
length, often a few sentences or a simple flow chart arrangement of 
words (similar to the "graphic-word display" used by Barron and Stone, 
1974). And second, it is usually presented in language with which the 
student is familiar. | 

An organizer, derived from an instructional context, therefore, is 
similar in intent to Ausubel's advance organizer but differs in two 
specific ways. Descriptively, an "instructional organizer" seems to be: 

Introductory material presented at the beginning of an 

instructional segment making explicit, often in diagramatic form, 

the relationships among concepts in the learning materials. It 

is intended for the student's use throughout the instructional 

sequence and must therefore be relatively short and use language 

which exists in the student's repertoire. 

It is argued, in this paper, that these two characteristics of length 
and language may be critical in determining the utility of organizers in 
prose material. Therefore, it is proposed in this study, to produce an 
"instructional organizer" as described and to compare its utility to the 


original (1962) organizer which Ausubel and his colleagues produced for 


use with the same text materials. The "operational rule" used in the 
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production of the graphic-word organizer is discussed in the materials 
production section on p. 31. The intent here, is not to determine whether 
organizers "provide subsumers in cognitive structure" or "tell the student 
where he is, where he's been and where he's going." Rather, an attempt 
will be made to determine whether the instructionally described organizer 
specifying a shorter and simpler language form will lead to improved 
performance scores in comparison with performance scores produced using 
the original organizer. 

In summary, since one of the problems associated with the use of 
advance organizers seems to have been the lack of operationalization, a 
typical advance organizer (Ausubel & Fitzgerald, 1962) will be compared 
with an alternate organizer which has been described and derived from an 
instructional perspective. 

Instructional Objectives 

Instructional objectives are variously described by Chaar proponents 
but whether referred to as instructional objectives, behavioral 
objectives, or performance objectives they usually have some character- 
istics in common. That is, objectives are generally statements about the 
learning task outcome, couched in terms which make apparent to the 
students what it is they will be expected to do. The most prevalent 
definitions (e.g., Popham & Baker, 1970) of objectives usually include 
reference to the performance expected, the conditions under which the 
performance will take place and the criterion for success (e.g., 100% or 
80% correct). 

Instructional objectives have received a great deal of attention 
over the past 10 years. In 1971, the Canadian Teachers' Federation 


produced a comprehensive bibliography of over 350 books, papers and 
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articles (cited in Geis, 1972). Additional reviews have been published by 
Ammons (1969), Geis (1972), Deuchastel and Merrill (1973) and Lawson 
(1974). In 1969, Ammons concluded that: 


First, the terms themselves have no universally accepted 
definition so discourse about objectives occurs on several levels 
of generality. Second, a statement of objectives or a recommended 
methodology for determining objectives is almost always couched 
in value terms, which renders empirical research in the classical 
sense difficult. Third, the question of what objectives ought to 
be sought has a history which dates from at least Plato. Fourth, 
pronouncements about objectives are more or less explicitly 
analyzed and justified opinions. Fifth, studies of a largely 
empirical nature in relation to objectives are few compared to 
ae number of statements based upon individual or group opinion. 

p. 908) 


Geis (1972) generally concurs\ with Ammon's (1969) conclusions but 
suggests that: 

-.- it would be a shame if the momentum and forward thrust started 

by Mager's little book (if the anxiety which may be the motive 


force of the movement toward more explicit description of 
education goals) were blunted or dissipated by total uncertainty. 


(p. 16) 
and further Peccuncade that: 
- much more research is called for to support and shape practices. 
Behavioral objectives allow research to occur; they provide 
unambiguous material for study. This is a benefit in itself. (p. 17) 
Deuchastel and Merrill (1973) consider objectives as having three 
main functions in terms of: 

a) direction for teaching and curriculum development. 

b) guidance in evaluation. 

ec) facilitation of learning. 

It is the third function, the facilitation of learning towards 
which their review and the goals of this research are directed. After 


reviewing 27 studies Deuchastel and Merrill conclude that results from 


studies comparing groups with and without objectives were at best 
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inconsistent with equal numbers obtaining and not obtaining differences 
between experimental and control groups. They considered the studies in 
terms of topic or subject matter, level of schooling, and time or duration 
of instruction. None of these factors produced any more consistency or 
clarity since successes and failures occurred with each of them. 

Deuchastel and Merrill (1973) express concern that the results of 
their review may be overgeneralized to affect the use of objectives in 
curriculum planning and evaluation when their review directly addressed 
the issue of objectives in the facilitation of learning. In reference to 
facilitation of learning, Deuchastel and Merrill (1973) recommend that 
even with the inconsistency obtained, the use of objectives apparently 
is non-damaging and should therefore continue. 

Lawson (1974) reviewed instructional objectives in terms of their 
role in promoting attending behavior, the degree of specificity of 
objectives and the effects of their location or placement. As in the 
other reviews cited he found the results equivocal. In werereces to the 
promoting of attending behaviors, Lawson (1974) concluded that objectives 
are helpful in promoting attending behaviors, and in situations where the 
objectives are salient to the learning task, do promote learning. He 
found the research relating to specific and general objectives so vari- 
able in details that neither comparisons nor conclusions could be made. 
In addition, he suggested the issues of timing and placement may account 
for differences obtained when using specific and general instructions. 
With reference to timing and placement, Lawson (1974) suggested that 
presentation of lists of objectives before relevant instruction may 
consist of an interpolated learning task thus inhibiting performance. 


Further, he suggested that interspersed objectives may provide "cue- 
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13 
producing stimuli for elicitation or shaping of efficient attending 
behavior which, in turn, enhance comprehension and retention." (p. 13) 

Summary and Critique 

There are a number of enduring problems associated with the use of 
instructional objectives to facilitate learning. As noted in each of 
the reviews discussed, the conclusions from research evidence must remain 
equivocal. The current status of research with instructional objectives 
provides little substantive support for the use of objectives in learning 
from prose instructional materials. In particular, few specific 
generalizations appear plausible in relation to the critical operational 
characteristics of objectives. 

There are several problems in evaluating research papers in the use 
of objectives to facilitate learning. First, there is the issue of 
definition. Objectives used in the papers reviewed by the authors 
previously cited vary from the well-operationalized statements which 
include behavioral indicators, specification of conditions, and 
specification of criteria through rule-like statements, vague general 
statements and specific reference to one text sentence. 

A second methodological problem in this literature is the manner of 
presentation. It is not always possible to determine whether the 
objectives were presented prior to instruction in list form, verbally or 
on paper, interspersed throughout instruction in a pre- or post-location 
in reference to the relevant instructional segment, or in a massed post- 
location. Any one of these variations and many other means of presenta- 
tion have occurred in the research described by the reviews. Very few 
authors appear to consider presentation mode an important issue. Only 


one (Papay, 1971) explicitly considered the massed/distribution issue. 
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A third, related issue was addressed by Kaplan and Rothkopf (1974) 
and Rothkopf and Billington (1975) who demonstrated the density (number 
of relevant text sentences in relation to number of objectives) is a 
factor which influences whether objectives may be achieved. Other than 
those cited papers, many authors don't discuss the relationships 
between presentation manner, massing or distributing, oral or written, 
pre- or post-presentation, or amount of text relevant to objectives. 
Whether all of these factors may have substantial influence of the 
relative efficacy of objectives for learning remains an empirical 
question. 

A fourth issue, which has received little attention is that of the 
structuring of the learning materials themselves. In many cases no 
examples or descriptions of the organization of the materials have been 
provided. However, in research where the instructional materials were 
explicitly organized and highly structured no support has been obtained 
for the use of objectives. (e.g., Etter, 1969; Merrill, 1970; Smith, 
1970; Merrill & Towle, 1971) This lack ei Veeteeragencca with highly 
structured materials may be the evidence that structural features of 
materials are important factors in the use of objectives. For example, 
it may be that when the materials themselves have been organized on the 
basis of objectives, then nothing further is gained by explicitly 
providing objectives. 

These four issues, operational definition, presentation type, 
density and structure of materials seem important considerations which if 
not directly investigated should at least be made explicit and control- 
led. Only the issue of operational definition, in terms of intended 


function will be directly addressed in this study. The other factors 
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will be described and controlled. 

An_Instructional View of Objectives 

The use of instructional objectives arises from an instructional 
context and is based essentially on an attempt to have teachers and 
other professionals make explicit behavioral statements which they will 
accept as evidence of a student's mastery of an objective. Great effort 
is made by proponents of objectives (e.g. Popham & Baker, 1970) to help 
the professional to produce and use non-trivial objectives. Their 
relative success in this respect has been demonstrated by the extensive 
support for and use of opjectives throughout many educational texts. 
This concern with non-trivial objectives and with making explicit the 
behavioral aspect of a student's response is not the characteristic of 
current research in the use of instructional objectives. Indeed, some 
research (e.g., Rothkopf & Kaplan, 1972) which carefully defines a 
"specific objective" as one which can be answered by one text sentence 
seems to be directed towards evaluating "trivial" objectives. At the 
same time, it essentially ignores the concerns with the student's 
response and the conditions under which it will be made which are the 
critical concerns for the production of objectives in the first place. 

The argument to be presented here is not basically directed to the 
practical use of objectives in instruction but specifically to obtaining 
empirical evidence about their usefulness. It seems necessary to deter- 
mine at the outset, what we intend objectives to do. If, for example, 
objectives are "orienting stimuli" as Rothkopf and Kap tein (1972) seems 
to believe, perhaps the most effective strategy would be to simply number 


sentences then say, "Look at sentence 10." 


The original point of the objectives movement, however, seems to have 
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16 
been to draw attention away from exclusive centering on content issues 
to considerations of "what should the student do."" The main claim for 
the utility of objectives presented to the student has been that they tell 
the student what he will be expected to do after the instruction. It 
seems useful, therefore, to consider objectives in terms of this critical 
feature and to determine whether it might be Heeb alineseeaany feature 
of objectives which are to be presented to students in order to improve 
performance. There is some evidence (e.g., Carey & Lockhart, 1973) that 
students who are told they will receive a recall test, perform differ- 
ently from students who are told they will receive a recognition test. 
Perhaps the prior information about what kind of test they will get is 
a critical component for objectives. 

A second related characteristic seems to be the information level 
of the test. For example, it may be useful if the student knows that he 
will be tested on single details (verbatim) or on several related details 
(paraphrase). 

In the research literature, there are a number of explanations to 
account for the lack of success in using objectives. One common explana- 
tion is that "the students didn't know how to use them.'' Several 
researchers (Kurtz, 1972; Morse & Tillman, 1972) have attempted to teach 
students how to use objectives, but with no improvement in performance. 

It seems plausible to hypothesize that even when an objective is "well 
written" it may not be useful to the student simply because it is 
presented before the student knows anything about the content, consequent- 
ly the student cannot make use of content references. If, however, a 
student is told that his objective is to answer a multiple-choice or a 


short-answer question which deals with specific details or with 
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relationships among concepts, this may be useful information to him even 
without his knowing the content in advance. It is proposed in this study 
to produce objectives which are entirely "content-free" and to compare 
their effectiveness with the effectiveness of traditional objectives 
which, while having a "task" or behavioral reference may have this refer- 
ence obscured in a content statement with which the student is unfamiliar. 

The intent is not to comprehensively evaluate the hypothesized 
functions of objectives. It is recognized that they may have many 
functions beyond the "task cue" function discussed here. However, it 
seems useful to attempt to isolate a crticial component and to assess its 
usefulness in relation to the traditional approach which has not, at this 
point, received a great deal of empirical support. 

In summary, the plan in this study is to prepare two types of 
objectives: First, objectives which have explicit content references 
(content-referenced); Second, objectives which specify the type of 
response (open-ended or multiple-choice) and the kind of information 
(verbatim or paraphrase). The latter are referred to as "content-free" 
objectives. 

Adjunct Questions 

The use of questions as aids to learning is not new and the research 
related to the use of inserted questions or test-like events is currently 
associated with Rothkopf and Frase who have published numerous papers in 
the area. Rothkopf (1970), for example, describes his interest in adjunct 
questions as a means of influencing "mathamagenic" behavior. Mathamagenic 
behavior is a term coined by Rothkopf to describe any activity, such as 
answering questions, which a learner engages in and which promotes or 


"sives rise to learning." According to Rothkopf (1970) these activities 
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consist of orientation, object acquisition and translation and processing. 
It is this latter, translation and processing function which is the 
primary role of inserted questions according to Rothkopf (1970). 

Rothkopf views adjunct questions as a means by which the experimenter 

can influence this translation and processing behavior thereby ensuring 
the student converts "nominal" into "effective stimuli." This position 
is consistent with Craik and Lockhart's (1972) "Levels of Processing" 
hypothesis which also postulates the importance of the qualitative inter- 
action of student and materials. Both these positions are consistent 
with the instructional interest in a mechanism or strategy which will 
support the students' learning processes. 

There are two basic claims made for the facilitation of learning by 
inserted question. First is the claim of "direct effects", that is, the 
effects of inserting a question within the text on performance on the 
same question in subsequent testing. This is usually referred to as a 
repeated or relevant test item. The second is the claim of "indirect 
effects", that is, the effects of inserting one question in the text on 
the performance on a different question on subsequent testing. The latter 
type of test item is usually referred to as a novel or irrelevant test 
question. 

The claims for direct and indirect effects of questions in learning 
from prose are closely associated with the issue of location or place- 
ment of questions in text. The most common arrangement for questions 
has been a pre- or post-location, that is, either preceding or following 
the text material. 

The issue of direct, indirect and location effects have been rather 


extensively considered in the research literature. Anderson and Biddle 
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(1975) after a thorough review of current research, conclude that there 
is substantial evidence for the direct effects of both pre- and post- 
questions. They also found evidence for smaller but consistent indirect 
effects of post-questions. They point out that generally, direct effects 
are four times greater than indirect effects. 

Both of these conclusions should, however, be considered in the 
light of other authors who were not represented in Anderson and Biddle's 
(1975) review. Specifically, Ladas (1973) criticizes: the evidence 
presented to support the claim of indirect effects of questions on two 
grounds. First, he criticizes Rothkopf (1966) and Rothkopf and Bibicos 
(1967) on the grounds that they employed a posteriori multiple totest. 
Second, Ladas (1973) points out two other papers (Frase, 1967; 1968) 
failed to include control groups. Subsequent data presented by Hiller 
(1974), Swenson and Kulhavy (1974) and Wilson, Koran and Koran (1974) 
also represent failure to support the indirect effect of questions on the 
learning of unquestioned material. 

Arguments against the claim for the direct effects of questions are 
somewhat less empirically based but nevertheless compelling. Carver 
(1972) addressed the problem of lack of control of time as a relevant 
variable. He pointed out that on the whole, questioned groups took more 
time than non-questioned qroups. He argues that Rothkopf's data does 
not contribute to the important issue of learning "efficiency" which 
relates to whether, if time was constant, the performance of the 
questioned group would still exceed the performance of the unquestioned 
group. Other researchers, for example, Peeck (1970), Morasky and Willcox 
(1970) and Corrozi (1971) found that when time with materials is control- 


led or equated performance differences between questioned and unquestioned 
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groups disappears. In addition, Carver (1972) expresses scepticism 
about Rothkopf's (1965; 1966) claims that questioning influences the 
students' learning strategies. Carver suggests that students in 
Rothkopf's experiments really had two sets of instructions, one the 
general prescription to "learn all the material", and the other implicit 
in specific detailed questions to "learn these specific details." Thus, 
Carver claims that the existence of the two kinds of instructions 
militates against Rothkopf's conclusion that it is the questions and 
not the general instruction which influences students processing. 

Summary and Critique 

Drawing conclusions on the basis of available evidence regarding 
the utility of questions in text is made difficult by a number of 
methodological problems. First, as with organizers and objectives, the 
range of materials and procedures which have been used is extensive and 
rarely reported in sufficient detail to make meaningful comparisons. 
Second, there has been a tendency to draw conclusions fate the existence 
of direct effects of questions on repeated test items without consider- 
ation of time as a contributing factor and without excluding the 
potential effect of general instructions to "learn all the material." 
The third problem in drawing conclusions from the literature is related 
to the claim for indirect effects of questions on new or irrelevant test 
items. Ladas' (1973) arguments and the lack of support from more recent 
literature makes the evidence for such effects tenuous. 

A more generic problem associated with drawing conclusions about 
the utility of inserting questions in prose is related to the research 
strategies generally employed. The majority of research using adjunct 


questions has employed an intentional/incidental paradigm and as DeDecco 
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24 
and Crawford (1974) point out: 

In an incidental learning situation, the experimenter does not 

provide instructions which prepare students for a test on the 

materials they are to learn. The students can be given no 
instructions at all or they can be directed to learn one and be 
tested on another task. Intentional learning provides for the 

prior announcement of the test and the learning task. (1974, p. 225) 

In general, the intentional/incidental paradigm which arranges 
conditions so students are told to learn one thing Bee eesa on another 
is a means of studying learning processes. Instruction, on the other 
hand, is intended specifically to promote learning. The concern in an 
instructional paradigm is primarily with learning outcomes rather than 
learning processes. Instructionally, the intentional/incidental paradigm 
may be viewed as a means of evaluating how questioning specific details 
promotes the learning of those aeeeLiE and not other details rather 
than as a means of determining whether or how inserted questions improve 
learning from prose. In this study the intent is to examine the learning 
outcomes associated with the manipulations rather than to examine the 
learning processes which may be involved. 

There are instructionally relevant and important questions to be 
asked in regard to the utility of inserting questions in text. These 
questions are related to issues such as type, location ee frequency of 
questions which will support the students' meaningful processing 
activities in learning from prose. 

The array of problems described above seems to make generalizations 
from research with adjunct questions difficult and possibly inapplicable 
to applied instruction. It should be remembered, however, that the 


evidence provided in Anderson and Biddle's (1975) review for a 


facilitation of post-questioning strategies on learning from prose cannot 
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22 
be discounted. The possible contribution of this evidence in the prepara- 
tion of prose instructional material will be discussed further in the 
following section of this paper. 

An Instructional View of Adjunct Questions 

In instructional contexts the use of questions inserted into text can 
be viewed as "practice", that is, an opportunity for the student to per- 
form the expected behavior prior to the test situation. There are three 
ways in which instructional "practice" seems to differ from the research 
usage of inserted questions. 

First, in research, the presence and location of the questions are 
often independent variables. In instructional procedures, questions as 
practice are almost always included and placed in a post-location. While 
there is some discussion of the necessity of including practice most 
instructionally prepared materials include such opportunity and assume 
that a learner who has no need for practice will simply omit it. 

(cf Merrill, 1975) 

Second, in research, the inserted questions rarely include feedback 
while in instructional procedures practice with feedback is the norm. 

The role of feedback has received considerable attention (e.g., Tobias, 
1973; Geis & Chapman, 1971) but the most common usage appears to be 
practice with feedback. 

The third difference appears in terms of "consistency." Consider- 
able effort is expended by instructional practitioners in maintaining a 
consistent relationship among objectives, questions, and test items. 

(cf Popham, 1973; Merrill, personal communication, 1976). Within 
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research contexts, however, this issue is viewed rather as "identical" | 


or "applied." 
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Specifically, instructional practitioners argue the importance of a 
relationship among objectives, questions and testing such that the 
behavior the learner is asked to perform, permitted to practice and 
subsequently tested upon, is consistent. In respect to textual informa- 
tion, for example, there are at least two dimensions along which 
objectives, practice and test items might be consistent or inconsistent. 
Objectives, questions and test items could be consistent or inconsistent 
in terms of the response type. To be consistent an objective which 
specifies multiple-choice response format should be accompanied by 
multiple-choice format practice and multiple-choice format testing. A 
second dimension along which inconsistency or poems eesics is possible is 
that of information level such as described by Bloom et al (1971). To 
be consistent in this dimension, objectives, questions and testing would 
all have to occur at the same taxonomic level. 

The "consistency" argument derived from instructional practices 
differs from the theoretical arguments presented in the 'mathamagenic" 
and "Levels of Processing" hypotheses in terms of the specification of 
type of question for various purposes. The "consistency" argument 
suggests that if the instructional intent is to produce "higher order" 
skills then the "higher order'’ skills should be practiced during the 
study period. If the intent is to produce factual information then the 
student should practice with factual information during the study period. 
There is no implicit assumption made that students who practice the 
"higher order" skills will automatically acquire those skills as well 
as related eae factual information. Rather, the assumption is 
made that the instructional developer will identify any necessary 


underlying factual information and explicitly provide for the practice of 
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24 
that information if it is necessary to the acquisition of the "higher 
order" skill. The "mathamagenic" and "Levels of Processing" hypotheses, 
on the other hand, lead to a position which assumes that practice on a 
"higher order" skill will lead automatically to greater acquisition of 
specific underlying facts. There is some research support for this 
position (e.g., Frase, 1971; Rickards & DiVesta, 1974). 

It is important to note that the positions described above represent 
views held by authors and theorists from divergent psychological tradi- 
tions. The instructional tradition is essentially behavioristic in nature 
and makes no explicit assumptions about the characteristics of the 
learning processes within the student. Rather, the behaviorist approach 
can be characterized as suggesting that whatever behavior is desired has 
to be taught. The ''mathamagenic" and "Levels of Processing" hypotheses 
arise from an essentially cognitive school of psychology and are concerned 
with making inferences about the characteristics of the internal state 
called learning. This approach is somewhat more sensitive to and inter- 
ested in the unplanned effects which are associated with student's 
cognitive processes. 

One specific comparison of questions used by researchers from the 
cognitive tradition (Anderson & Biddle, 1975) is the verbatim/paraphrase 
type. The argument they present suggests that a paraphrase question 
requires broader processing (both semantic context and lexical form) 
while verbatim questions require only the processing of the same lexical 
form. Then the insertion of verbatim items should lead to less learning 
than insertion of paraphrase items. Since the positions suggested by 
"consistency" and "mathamagenic" or "Levels of Processing" hypotheses 


lead to slightly different suggestions for the use of inserted questions 
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this research will attempt to examine empirically the effects produced 
using both positions. 

In arranging objectives, questions and test items consistently, two 
dimensions will be considered, the item forms (open-ended or multiple- 
choice) and the question type (verbatim or paraphrase). Consistent 
conditions will be provided with objectives, practice questions and test 
questions which match both in terms of item form and question type. In- 
consistent conditions will be arranged so that they meet as far as 
possible, the conditions which are suggested by the "mathamagenic" and 
"Levels of Processing" hypotheses. Specifically they will receive only 
paraphrase questions inserted in the text while they will be tested on 
both verbatim and paraphrase test items. All inserted items for these 
groups will be of an open-ended format, a type of question which, accord- 
ing to Anderson and Biddle (1975), leads to the largest direct effects. 
It should be noted that since this research deals with objectives as 
well as questions and since objectives are necessary in consistency 
arrangements, objectives will be present in the text of inconsistent 
groups. The typical research relating to adjunct questions (e.g., 
Rothkopf, 1970; Frase, 1970; Anderson, 1970) does not include objectives 
with questions. In order to retain comparability among groups the 
inconsistent questions groups will be provided with objectives which are 
consistent with the inserted questions. The inconsistency, therefore, 
is in reference only to the relationship between practice and test items. 

In summary, there are four ways in which instructional practice 
appears to differ from research with adjunct questions. First, practice 
is almost always included and follows the relevant material--it is an 


integral part of instruction rather than something which is occasionally 
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inserted. Second, corcern with the effects of practice is explicitly 
"intentional." That is, the intentional/incidental paradigm is not used. 
Third, feedback in some form is usually included. Fourth, practice must 
be consistent with both objectives and subsequent testing. 

In this study, three of the four differences will be held constant. 
Practice will be used, an intentional paradigm will be employed and feed- 
back will be included. Only one feature of practice will be manipulated, 
that is consistency. 

In order to investigate the contribution of consistency to the 
effects of practice on performance scores, two arrangements will be used. 
First, an inconsistent condition will be established by inserting 
questions which differ from test items in terms of the two dimensions, 
form (open-ended and multiple-choice) and question type (verbatim and 
paraphrase). The second, consistency arrangement will involve inserted 
questions, objectives and test items which match on both dimensions. No 
inserted questions will be identical to test questions. 

The Research Plan 

Within the context of applied instruction, it is not uncommon to 
find several general prescriptions for learning from text. First, it is 
recommended that the student be provided with some information about where 
he is in the material, where he is going and where he has been. Second, 
it is suggested that the student be told what he is to do with the 
information he is given. Third, it is recommended that the student be 
given a chance to practice whatever it is that he is expected to do. 

These three general prescriptions appear to correspond with the three 
manipulations organizers, objectives and questions. The use of these 


manipulations and the existence of the prescriptions is visible, not in 


1 ceuntith a 
8 dan ssbb de 


hea 2) My pee. 
inetiatoo bist oi) Lite ssogoatn #uei adj 25° 8 ia: bik Tale 


Ht 
Wie 
a. oe st 


~best Boe be vofams ad. iliv ag bivreg, banotsasvay ‘ak 


be telaqiiesim ad) it iw sop aera he ono fA 


Me a4 sig od yonedelenos to MOK Mtr acs |f3 wih i iil 


puaw od (ily agvemegnstss awit aeyoon Syoemscd rey) ah Sasa dole 


guiviasnt yd bsrterl isdee o¢ Lliw notpibues is. cha sik: 
F 


enfoteremibh) owl sds to ABS | ing omg? dandy mat —* 


bas sikentivan). egys aakte2 anil i i. i a algt ai bores 


badaoént svl@yvat (EF jaa IAG th | <ouitlatega Si oH 
“Of @abbensokWodgéd tro! 4oabe ots baw Nacht fe54 ote wat a 
Janoivesup BOT) Od, Leatihab! Byes KavidaoR Seeup boa 
: ' galtinogastiow att! i a aed a 

ad noms doh et si ,aoksspyvedtibatiqgs tb Saal aol a2 sbdadw 
ak 3 jsighetin S59 612 Gninxaed 502 aféttq} sNease [ayenay ievoves bar 
stotlwy Juefs Holttemroint ome dite bebiverg 66 csbue six aade dobermans 
baouse’ .n3ed aad ed oisiw bosigatog at of sioi p Lepage ene rato ed 
oid Nthw 05 02 ef od te bids 3s¢ sasbuie St seng lai ak st 
sd aushpta eft -geds Bah sinedeer, @kett SEAT laoul@ 2k sd pnttdeipaleanes 7 
(G8 C2 Delssqyse st ad 25de 2F Si xovs dady Sottotiq 02 sores Be perks 

aetiid Sy{i) vidiw beoqesxi0s i ROGGE Wedtcak bibs dansil taxon sytda sged® *, 
rent ae em og? <anbigeenp Sas asvisostdt heel teat nntinees 

nk tnt ,eldtety ef enolsqtases34 od7 to boas tee “og bap ate 


aa 
research literature but rather in instructional products (e.g., 
Courseware Inc., 1974). 

This is a substantial deviation from the procedures which are 
followed in research using organizers, objectives and questions where 
each of these manipulations appears to be viewed as an alternate serving 
the same function. Rothkopf and Kaplan (1972) for example, view their 
objectives as essentially declarative forms of questions and both 
manipulations are seen as mechanisms to "maintain attention to text." 

The issue of whether the three manipulations can be used as alter- 
nates or whether they are three different and important manipulations 
does not appear to have heen investigated. 

Each of the manipulations, organizers, objectives and questions has 
been prepared in two ways. The first, the way in which the manipulation 
is most commonly used in the research literature. The second, the way 
which seems to be most appropriate to instructional prescription and 
usage. All three manipulations, in one or the other form, will be 
included in each instructional package presented to the students in order 
to support learning from textual material. 

An attempt is made, in this research, to include some methodological 
controls which may have contributed to the lack of conclusiveness in the 
research data previosuly cited. First, each of the manipulations used is 
described and presented with the rules used in its production or 
acquisition. Second, the presentation mode most frequently used with 
each manipulation is adopted for use here. Advance organizers are massed. 
Objectives are dispersed preceding the materials to be learned. While 
in some cases objectives have been presented in a list form preceding 


the instruction, one of the common current uses is in a dispersed mode 
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(cf Popham & Baker, 1970). Inserted questions with feedback are dis- 
persed after the relevant textual materials. Third, time is controlled 
and identical for all students. 

It is hoped that these controls, in conjunction with an instructional 
approach to the use of organizers, objectives and questions will provide 
information useful in the preparation of prose materials from which 
learning is improved. (cf Merrill & Boutwell, 1973) 

The Research Question and Hypotheses 

Improvements in learning from prose materials can be accomplished by 
means of "instruction" which, according to Gagne (1974) is the arrange- 
ment of environmental conditions which support the learner's processing 
activities. Careful consideration of the research relating to three 
manipulations has led to the conclusion that the effectiveness of 
organizers, objectives and to a lesser extent, questions, in providing 
such support is equivocal. 

Each of the three manipulations were considered in terms of current 
instructional precepts and procedures and then restructured to suit 
their use in applied contexts. It is hope that the use of these manipula- 
tions in their two different forms will lead to improved learning from 
prose. 

In order to empirically determine the usefulness of these manipula- 
tions in improving learning from prose, an experiment has been conceived 
to explore one general question: 

Are there differences in students’ learning with organizers, 

objectives and questions as they are found in research when 


compared to organizers, objectives and questions developed from an 


instructional context? 
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From this general question four hypotheses have been derived for 


testing within an experimental design: 


H. 1: A prose organizer produced by Ausubel and Fitzgerald (1962) 
will lead to significantly lower performance scores than 
a graphic-word organizer. 


H. 2: Content-referenced objectives will lead to significantly 
lower performance scores than content-free objectives. 


H. 3: =%Inconsistency between inserted questions and test questions 
will lead to significantly lower performance scores than - 
consistency between objectives, inserted and test questions. 


H. 4: The use of organizers, objectives and questions with text 
materials will lead to significantly higher performance 
scores than will occur with text alone or manipulations 
without text. 


A fifth more general and exploratory hypothesis is added. 
H. 5: Some differences exist between performance scores of 


experimental and control groups on immediate and delayed 
testing. 
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CHAPTER IT 


Method 

The material used in this research, with the exception of the basic 
text produced by Ausubel and Fitzgerald (1962), was produced by this author. 
A brief description of the plan and preparation of the materials follows. 

Text 

The prose material used is a slightly shortened version of the 
original text produced by Ausubel and Fitzgerald (1962). Shortening was 
done in response to students' difficulty in completing the material in the 
time available during a pilot study (approximately 2500 words). 

This text was selected because: 1) It has been used previously in 
similar research (Ausubel & Fitzgerald, 1962; Papay, 1971); 2) Both 
cited authors suggest that it is comparable to university level texts; 

3) The text is considered by the cited authors to be appropriately 
difficult and unfamiliar for research use. 

The text was subdivided into 13 sections of approximately two 
paragraphs each. One section consisted of one long loi alae and two 
sections consisted of three short paragraphs. The sections were typed 
on white 8 1/2 x 11 sheets of paper with space left at the top for 
objectives and at the bottom for questions (see Appendix Le 

Advance Organizers 

Two types of organizers were used. The first, in Table 1 as level 
one, is a 450 word prose passage produced by Ausubel and Fitzgerald (1962) 
and is considered by them to be at a “higher level of generality and 


inclusiveness" than the accompanying text (see Appendix III). 
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The second type of organizer, the graphic-word organizer, is a one 
page visual-verbal combination produced by drawing the main idea from 
each paragraph of the original organizer and placing these main ideas in 
a hierarchical arrangement. Wherever possible a less technical term was 
used. These main ideas were then connected by a system of encircling 
lines and arrows to indicate relationships among the ideas (see 
Appendix III). 

Instructional Objectives 

Two types of objectives were prepared for this study. Content- 
referenced objectives are similar to objectives found in the research 
literature and variously designated "specific objectives", "behavioral 
objectives" or "performance objectives."" They consist of statements 
directing students to learn specific information in the text (see 
Appendix IV). 

Content-free objectives were prepared following the discussion on 
page 15ff. They direct the student to the relevant type of information, 
specific detail or several details combined, without including any 
statements drawn from the content of the text. In addition, they specify 
the type of response which will be expected, open-ended or multiple- 
choice (see Appendix IV). 

Adjunct Questions 

Questions for consistent groups were all multiple-choice items 
prepared according to the following two rules: 


Verbatim Items: 


One specific detail of information is taken from the prose 
such that the question stem and its correct alternative 
form a verbatim statement found in the text. 
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Paraphrase Items: 
Two or more specific details found in non-adjacent text 
sentences are combined such that the stem and the correct 
alternative is a paraphrase non-identical to the sentences 
as they appeared in the text. 
Six verbatim multiple-choice items and seven paraphrase multiple-choice 
items were prepared for insertion into appropriate text sections. 
Questions for inconsistent groups were all open-ended paraphrase 
items prepared using the paraphrase rule. All inserted questions were 
placed on the bottom of the page of text to which they referred. 
Answers, in sentence form, were presented for all questions on the page 
immediately following the relevant text. Questions and answers are 
presented in Appendix V. 
Achievement Tests 
Twenty-six multiple-choice test items were prepared. Twelve of 
these items were verbatim questions for text sections which had received 
inserted verbatim items (consistency groups). Fourteen items were 
paraphrase items for the text sections which had received inserted 
paraphrase questions (both consistency and inconsistency groups). All 
test items were five alternative multiple-choice format. No test items 
were identical to inserted questions. 
To provide for supervision and easier monitoring tests were prepared 
using different paper colors and different item orders for immediate 


testing. Different paper color and different ordering of both items and 


the alternatives were used for delayed testing. 


Questionnaire 


A short questionnaire asking students opinions and attitudes towards 


the research was prepared (see Appendix 5 at Gr OB 
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Subjects 

The subjects were students registered for spring session classes in 
the Faculty of Education and students registered in "university level 
programs" at Grant MacEwan College. They were all volunteers with 
university experiences ranging from 1 to 4 years. There were approxi- 
mately equal numbers of males and females. The total sample size was 
132, 12 students in each of 11 groups. 
Materials 

Packages of materials for all groups were prepared prior to the 
first meeting with students. Each package contained: 

a) A general introduction (see Appendix VIII). 

b) Learning materials consisting of the appropriate combination 

of organizer, objectives, questions and text. 

c) An immediate post-test with machine scoreable IBM answer form. 

d) A questionnaire. 
Procedures 

All experimental sessions occurred during regular classroom time. 
A short introduction was presented verbally to the group (see Appendix 
VIII). A short pause was allowed for students to leave if they wished. 
The number of students opting to leave varied from 1% to 60% of the 
group. Randomly arranged packages of materials were then distributed to 
the students. In each group an attempt was made to ensure at least one 
package from every experimental and control group was distributed. 

Students were asked to remove the white learning materials from 
their envelopes and told they had 35 minutes in which to study before 


they would be tested. 


Time available was recorded on the blackboard every 5 minutes and 
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34 
announced every 10 minutes. After the 35 minute study time had elapsed 
students were asked to mark on their materials where they stopped if 
they had not completed them. They were then asked to replace the study 
materials in the envelopes and to take out the test paper and answer 
form. 

Students were told they had 20 minutes in which to complete the 
test. After its completion they were to fill out the questionnaire and 
return all materials to the experimenter. 

Ten groups of students ranging in size from 6 to 70 participated in 
the research in this fashion. After every session the experimenter 
examined each returned package. Thirty students were removed from the 
experiment for one or the other of the following reasons: 

a) failure to complete the study materials, 

b) failure to complete the test in the allotted time. 

These deletions occurred approximately equally in all experimental and 
control groups. 

Whenever a student's materials were removed from the experiment, a 
new package for the group in which they had participated was added to 
those not yet completed. In this way, equal numbers of students in each 
group was ensured. 

Approximately one week after the initial session the experimenter 
returned to the classroom for delayed testing. At this time, the students 
were told that no forewarning had been given in order to prevent undue 
anxiety or outside study (see Appendix VIII). The students were then 
asked to write the delayed form of the achievement test. 

After 12 students had worked through materials for each of the 


eleven groups, IBM answer sheets were checked for completeness. Double 
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marking or blanks were treated as errors. 

The answer sheets were then submitted for machine scoring and card 
production. Two cards were produced for each student who had partici- 
pated in both immediate and delayed testing. One card was produced for 
students who participated in immediate testing only. 

Three scores were punched on each card: 

a) Subscore "V'"' —- the number of correct responses on verbatim 

items (total of 12). 
b) Subscore "P'" - the number of correct responses on paraphrase 
items (total of 14); 
c) Total score - the number of correct responses on all items 
(total-ofv 26) 2 
Research Design 

A 2x 2x 2 factorial design was employed in which each factor, A 
(organizer), B (objectives) and C (questions), had two "levels." The 
levels represent the two forms of each manipulation. This design 
produced a factorial arrangement of eight groups. In addition, three 
"control" groups were added to the design. The "text only", "old 
manipulations only" and "new manipulations only" are described in Table 1 
which presents the variables used in this study. A list and verbal 
descriptions of all experimental and control groups is presented in 


Appendix lI. 
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CHAPTER IIT 


Results and Discussion 

Several types of analyses were performed using the three dependent 
measures (verbatim (V), paraphrase (P), and Total scores). First, 
three-way analyses of variance were performed using the data from the 
eight experimental groups within the factorial design. Second, one-way 
analyses of variance were calculated using the eight experimental and 
three control groups. Third, two-way analyses of variance were performed 
upon the immediate and delayed test scores for both experimental and 
control groups. The results of these analyses will be presented 
individually and discussed in terms of their contributions to the 
research hypotheses. Since each type of analysis has some bearing on 
several hypotheses discussion will proceed in chronological order 
according to the hypotheses involved. | 

Throughout the discussion issues relating to the relative magnitude 
of scores will arise. Therefore, Table 2, providing means V, P and 
Total scores for all groups on immediate and delayed testing, is inserted 


here for continuing reference. 


Hypothesis One 


A prose organizer produced by Ausubel and Fitzgerald (1962) 
leads to significantly lower performance scores than a graphic- 


word organizer. 


Tables 3, 4 and 5 present the results of three-way analyses of 
variance performed upon Verbatim, Paraphrase and Total scores for the 
eight experimental groups. Tables 3 and 5 indicate a significant 
main effect for organizers in the analyses upon V and Total scores, 


F(1,88)=3.82, p¢.05; F(1,88)=5.01, p<.05. Results presented in Table 4 
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Table 3 


Analysis of Variance of Verbatim Scores 
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Table 4 


Analysis of Variance of Paraphrase Scores 
for Eight Experimental Groups 


Source 
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Table 5 


Analysis of Variance of Total Scores 


for Eight Experimental Groups 
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indicate there were no significant main or interactive effects for 
organizers on P scores. These results reverse hypothesis one by indicat- 
ing a superiority of the prose over the graphic-word organizer. 

These data represent a reversal of the predicted differences. In 
the discussion (p. 8 ff) it was argued the graphic-word organizer 
should lead to better learning than the prose organizer. This expecta- 
tion was based upon the applied experiences of groups such as Courseware 
Inc. and the logical argument that students would be better able to use a 
manipulation which was more parsimonious and less technical. Support for 
this position was also drawn from research reported by Barron and Stone 
(1974). There is, however, one way in which Barron and Stone (1974) 
used graphic-word organizers which differs from the use in this study. 
Their students participated in its production, either with the experi- 
menter or with other students. This difference was noted at the outset 
but no information was available to indicate the relative importance of 
this "interactive" component in learning from prose material. The 
possibility that "student generated" organizers may be more effective 
than "author generated" organizers has also been discussed by Dooling 
and Mullet (1973). Lack of student participation or interaction, there- 
fore, may have contributed to the relatively poor performance of the 
graphic-word organizer. Wittrock (1974) discusses the importance of the 
learner's role in interpreting and processing of stimuli and draws 
attention to the importance of the learner's previous experiences. This 
factor cannot account for the relative effectiveness of the prose organ- 
izer since it too was "author generated." It is necessary to look 
elsewhere for an explanation which will account for this superiority of 


the prose over the graphic-word organizer. Such an explanation may be 
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found in considering relative familiarity and ability of students with 
verbal as compared to visual information. Holliday (1976) examined the 
effects of pictoral, block-word, and prose presentations with high school 
students learning verbal chains. He found that students could, when no 
alternatives were presented, gain information from all three presentation 
modes. But when pictoral or block-word formats were presented in 
conjunction with prose materials the students showed a marked tendency to 
prefer the verbal to the visual information source. In discussing these 
results Holliday (1976) suggests that students’ previous learning 
experiences have trained them to prefer prose over visuals as information 
sources. 

That students may be unable to extract meaning from the visual 
component of the graphic-word organizer is consistent with Salomon's 
(1974) suggestion that there are differences in symbolic codes in differ- 
ent media. In this case, the verbal code implicit in the prose organizer 
is one with which students have some familiarity while the visual code 
in the graphic-word organizer was probably relatively unfamiliar. Since 
this particular visual was unlikely to be "image producing" in its 
effect, it is possible that it had an interfering effect on the usual 
contiguous sequential encoding of the verbal material. 

This is a particularly important issue for instructional practice 
since the question of what visuals to use and how to use them is a 
common problem. The evidence from this research is consistent with 
Holliday's (1976) results and warrants further investigation. This issue 
may be one for which the use of an Aptitude-Treatment—Interaction (ATI) 
framework is most appropriate. Disordinal interactions have been demon- 


strated in an aural/visual modality framework by Ingersoll and Divesta 
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(1972). The issue of ability to use verbally or visually presented 
information could, for example, be addressed through the use of students 
who are: (a) verbally skilled (e.g., English majors or students scoring 
high on measures of verbal fluency), and (b) visually skilled (e.g., Art 
majors or students scoring high on visual ability measures) ; providing 
such students with either verbal or visual presentations might permit the 
detection of an aptitude factor relevant to the preparation of instruc- 
tional materials. In addition it seems possible to question whether 
students with one type of ability could be trained to use the other 
information source. 

Summary 

The data analysis provided evidence which does not support hypo- 
thesis one and is a reversal of the expected direction. The superiority 
of the prose over the graphic-word organizer was discussed in terms of the 
possibly important lack of interactive component of the latter and the 
possibly strong preference for and abdasl toe selich the former. These 


results were noted as requiring special attention for instructional pro- 


duct developers. 


Hypotheses Two and Three 


Content-referenced objectives lead to significantly lower 
performance scores than content-free objectives. 


Inconsistency between inserted questions and test questions 
leads to significantly lower performances than consistency 
between objectives, inserted questions and test questions. 


Results presented in tables 3, 4 and 5, showing no significant main 
or interaction effects for objectives and questions, represents a failure 


to support hypotheses two and three. In addition, relatively small 


mean squares indicate very little variance attributable to objectives or 
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45 
questions of either type. 

These results are somewhat surprising since both objectives and 
inserted questions were found in the literature to have contributed 
substantially to performance scores in many experiments. The lack of 
significant effects for objectives and questions seems most plausibly 
explained in terms of the evidence for their lack of use. 

The theoretical premises underlying the use of inserted manipula- 
tions in prose text are derived, in part, from discussions of the use of 
inserted questions and orienting stimuli generally. This position is 
presented by Kumar (1975) in which he describes the action of orienting 
stimuli as involving a two stage orienting response. Stage one is 
thought to be a brief phase involving gross motor responses while stage 
two is thought to be a more prolonged "tonic" phase involving scanning of 
details. The underlying assumption thus suggests that stimuli which 
elicit gross motor orinetation plus the scanning for details should 
lead to improved performance in relation to the information scanned. 

In his own research using out-of-context words and instructions to 
"note" or "remember", Kumar (1972) obtained relatively small differences 
in performance scores when compared to control group scores and no 
difference after 24 hours delayed testing. He discusses, in his explana- 
tion for these negative results, evidence from students that they often 
did notice the orienting stimuli and were aware, in the case of the "note" 
and "remember" stimuli, that they were expected to remember that informa- 
tion. The students, however, were in some cases explicit in their 
statements that they did not like memorizing or that there was too much 
information for the time given. Kumar (1975) thus concluded that while 


the orienting stimuli he used may have produced both stages of the 
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46 
desired orienting response, that such a response may well be insufficient 
for performance scores to be enhanced. He Suggests that in addition, the 
orienting response elicited may also need to be accompanied by the 
student's decision to retain the information given. 

In this research there is also some evidence to suggest that the 
students may have noted and even scanned the information. Over 70% of 
the students reported on their post-test questionnaires that they had 
used "all" or "almost all" the manipulations presented to them. However, 
examination of their learning materials packages revealed that less than 
30% of the students had marked anything on their packages beyond their 
names which were often pseudonyms. This may be taken as support of the 
argument that they did not form an intent to learn when it is viewed 
with the questionnaire evidence that over 70% of students said that when 
studying prose materials for learning they underlined, took notes, and 
answered questions. 

The decision or intent to learn is, of course, viewed by most 
theorists and practitioners as important to the learning process and an 
essential component of motivation. However, it is also true that 
motivational variables are usually treated lightly or ignored in the 
kind of research cited in the literature review. Most often motivational 
variables are not explicitly discussed or addressed, but some evidence 
relating to motivational variables may be drived from the description of 
students. For example, it is not uncommon to find that students are 
drawn from classes where "course credit" is granted for participation in 
research and where participation in at least one research project is 
mandatory for all students. In some cases, however, students were 


encouraged by their classroom teacher to participate and in other cases 
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47 
the classroom teacher was present at the beginning of the study. While 
most students appeared to be trying to study the materials and were 
cooperative, some did leave before and during the study time and a few 
expressed negative feelings towards the research being conducted during 
classroom time in a course which was very short and intense. It is not 
possible to draw firm conclusions relative to motivational states of 
students participating in this research; however, it seems reasonable to 
assume their motivational levels were similar to those in similar 
research contexts. This assumption is supported by the low average 
scores obtained by students in this research, ranging from 30% - 50%, 
which are comparable to scores obtained by Papay (1971) with similar 
materials. 

A differential effect of this hypothesized motivational state on 
objectives and questions in comparison with organizers could be related 
to the fact that organizers appeared in an initial massed position while 
both objectives and questions were distributed throughout the learning 
passage with objectives preceding and questions following the relevant 
text passages. It seems likely, that if motivational factors were 
involved they would have greater impact on the manipulations which were 
experienced over a relatively long period of time (35 minutes) than the 
manipulation which was experienced within the first few minutes. 

The relatively small effects of objectives and questions makes 
consideration of the hypotheses two and three speculative. However, it 
is possible to consider the trends which appear in the data. Figure 1 
presents the total score means for groups using the two types of 


organizers, objectives and questions. 


While the magnitude of the differences between the two types of 
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Figure 1 Total Score means obtained by groups using two types* of organizers, objectives 
and questions. 
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49 
manipulations is small, it is also obvious that one type (the "common 
usage" version) is always better than the other type. In the case of 
organizers, the difference is significant. The direction of this pattern 
is the reverse of initial expectations and is so consistent that it seems 
to require further conedieeeuion: In each case, one type of manipulation 
was based on common usage in the literature while the other type was 
drawn from applied contexts. Since instructional products development 
is currently more characteristic of business and industry than education, 
if students had any previous experience with the manipulations it would 
have been in the traditional or research form. The relevance of previous 
experience with objectives was mentioned in the literature review. In 
addition, Bassett and Kibler (1975) have recently demonstrated that 
training in the use of objectives can improve performance. The marginal 
superiority of content-referenced objectives over content-free objectives 
may indicate that students given content-referenced objectives did know 
how to use them while students given content-free objectives did not. 

A similar argument cannot account for the tendency for inconsistent 
questions to exceed consistent questions since the students would not 
have been aware of the dimensions along which the items were consistent 
or inconsistent and there is no a priori reason to assume more experience 
with open-ended than multiple-choice type items. While there is no 
statistically significant support, the trends in the data tend to favor 
the "mathamagenic" and "Level of Processing'' hypotheses since the 
inconsistent questions should have lead to greater processing activities 
on the part of the students. A "Level of Processing" hypothesis would 


predict that the more active task would lead to deeper processing and 


subsequent improvement in performance. (Craik & Lockhart, 1972; 
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Mistler-Lachman, 1974). This explanation is particularly plausible when 
it is remembered that while the inconsistent items were inconsistent at 
the "micro" level, that is, in terms of the specific test questions, they 
were consistent at a "molar" level in that they were drawn directly from 
the paragraphs to be tested. 

It remains, however, for subsequent research in which "real" 
students and "real" materials are used and in which some “intent. to learn" 
can be established to determine whether the trends visible in this data 
would become significant or indeed reverse themselves as was hypothesized. 

Summary 

The lack of evidence to support hypotheses two and three was dis- 
cussed in terms of the lack of evidence that either objectives or 
questions were used by students. The slight trends in the data which, 
tended to reverse expectations were noted and discussed. Possible 
explanations in terms of relative familiarity and depth of processing were 
suggested. 


Hypothesis Four 


The use of organizers, objectives and questions with text materials 
will lead to significantly higher performance scores than will 
occur with text alone or manipulations without text. 

Since the initial analyses demonstrated significant main effects 
only for organizers it was decided to collapse across this variable for 
analyses of experimental and control group effects. The four experi- 
mental groups receiving prose organizers were, therefore, combined into 
one large group (Prose) while the four groups receiving the graphic-word 
organizer were combined into another single group (G-W). Subsequent 
analyses were performed with random samples drawn from these two larger 


groups in order to avoid the problems of unequal "n'' and non-homogeneity 
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51 
of variance. 

The analysis presented in Table 6 is based upon Verbatim, Paraphrase 
and Total scores for 12 students randomly drawn from the "Prose" group 
and 12 students from the "G-W" group as well as the 12 pulddate in 
control group "old manipulations only." Significant differences were 
found in the analyses based on P and Total scores, F(2,33)=7.06, 

620053). (2,33)=7.023 p<.005. Table 7 presents the results of analyses 
using "Prose", "G-W'', and "text only" groups with significance obtained 
in the analyses using P and Total scores, F(2,33)=3.53, p<.05; 
F(2,33)=4.26, p¢€.05. Table 8 presents the results of the analyses using 
"Prose", "G-W'', and "new manipulations only" groups with significant 
effects from the analysis upon Total scores, F(2,33)=4.59, p¢.05. 

The results presented in Tables 6, 7, and 8 provide partial support 
for hypothesis four. In addition, from those results it can be argued 
that organizers, objectives and questions combined with this particular 
text lead to improvements essentially on paraphrase questions without 
showing marked effects on verbatim questions. 

Table 9 presents a Newman-Keuls comparison of the means of the 
"Prose", "G-W'', and control groups involved in the preceding analyses. 
Significant differences were found between the Total score means of 
"Prose" groups and the means of all three control groups. 

From the above results it can be concluded that some experimental 
groups achieved higher total scores than the control groups. This seems 
to have been a result of somewhat greater scores on the paraphrase rather 
than the verbatim items. This support of hypothesis four should be viewed 
with reference to Carver's (1972) argument that if all students are given 


equal time to study the material, students with text alone will perform as 
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Table 6 


Analyses of Variance of Verbatim, 


Paraphrase and Total Scores r, 
for Two Experimental Groups and One Control Group 


Scores Source SS MS df F-Ratio 
V Groups 16.89 8.44 Z Drala 
Error 128.67 ong0 33 

Chi Squared = 3.58, p=.17 

B Groups Soa 2 18.86 2 ia06 
Error 88.17 PH | 33 
Chi Squared = 2.74, p=.25 

Total Groups 104.16 52.08 2 7.02 
Error 244.84 FRa2 33 
Chi Squared = 2.79, p=.25 
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Probability 


NS 


.005 


.005 


2 Students were drawn randomly, 12 from each of the "Prose" and "G-W" 


groups. 
Control group was - traditional manipulations only. 
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Table 7 


Analyses of Variance of Verbatim, 
Paraphrase and Total Scores é 
for Two Experimental Groups and One Control Group 


Scores Source Ss MS df F-Ratio Probability 
V Groups 18.06 9:03 2 2:28 NS 
Error 130.50 By9D ee 
b 


Chi Squared = 9.80, p=.007* 


P Groups 20.39 10.19 2 She )) =O nan 
Error 95.25 2.09 33 


Chi Squared = 1.81, p=.41l 


Total Groups RO, 22 38.011 2 4.26 OO: 
Error 295.42 8.95 33 
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Chi Squared 


2 Students were drawn randomly, 12 from each of the ''Prose" and "G-W" 


groups. 
Control group was - text only. 


: Chi Squared test of homogeneity of variance. 
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Table 8 
Analyses of Variance of Verbatim, 


Paraphrase and Total Scores 
for Two Experimental Groups and One Control Group” 


Scores source SS MS df F-Ratio Probability 
V Groups 1472 5.86 2 2.19 NS 
Error 885 17 2367 38 
chi Squared = .14, p 93 
P Groups Fan 10.58 2 2.97 NS 
Error i i ba fees Sie Ko} 33 


Chi Squared = 3.40, p = .18 


Total Groups 64.39 32:19 2 4.59 305, 
Error ae a 7200 33 
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Chi Squared 


4 students were drawn randomly, 12 from each of the "Prose" and "G-W" 
groups. 
Control group was - new manipulations only. 

b 


Chi Squared test of homogeneity of variance. 
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Table 9 


Newman-Keuls Comparison of Differences Between Means 
on P and Total Scores 
for Two Experimental and Three Control Groups 


Groups Differences Critical Values 


Prose: G-W: Old Manipulations Only 


P Score Means 
Prose: Old Manipulations Only 2.50% [565 
G-W: Old Manipulations Only NS 


Total Score Means 
Prose: Old Manipulations Only 4.17% 2.74 
G-W: Old Manipulations Only NS 


Prose: G-W: Text Only 


P Score Means 
Prose: Text Only 1.83% eee) al 
G-W: Text Only NS 


Total Score Means 
Prose: Text Only 3.50% 3.01 
G-W: Text Only NS 


Prose: G-W: New Manipulations Only 


P Score Means 
Prose: New Manipulations Only LY67 1.90 
G-W: New Manipulations Only NS 


Total Score Means 
Prose: New Manipulations Only 2.92% 267 
G-W: New Manipulations Only NS 
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well as students with text and questions. Since equal time was available 
to groups with manipulations (experimental) and the group with text only 
this data does not support Carver's argument. 

It should be noted that Carver's argument involved the idea of 
"learning efficiency" and the small magnitude of difference scores 
obtained requires additional evidence to support or refute his position. 
Specifically, the question of relative efficiency of time spent is probably 
considered more usefully in the context of "meaningful" rather than 
"significant" differences. It would be useful, for example, to reconsider 
the question of level of performance with the use of instructional 
manipulations as compared to performance scores in the absence of the 
manipulations, in a more realistic context. In this research, the 
potentially critical motivational factors may not have been available 
because of the "experimental" nature of the research. In some respects, 
however, the research was a closer approximation of "normal" study 
contexts than is true of the research Carver (1972) was criticizing. 
Specifically, Carver addressed himself to papers by Rothkopf (1965; 1966) 
and Frase et al (1970) in which students were permitted oe ee one page 
at a time and not allowed to return or reread any of the material. In 
this study, students were given, and indeed took the option to read 
through and return or reread at their discretion. There were, of course, 
individual differences in the extent to which students exercised this 
option. However, the availability of the option to reread makes the 
study more naturalistic and in this respect the results could be viewed 
as evidence against Carver's position. 

In addition, the relatively low scores in all groups may be accounted 


for in terms of the time permitted. Thirty-five minutes may not have been 
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sufficient for students in some groups to read the organizer and the 
passage. During the data gathering phase it was observed, for example, 
that some students appeared to finish and simply sit until the study 
time had elapsed. Many of these students were in the text only control 
groups. Only rarely was an experimental group student observed to stop 
working before the study time had elapsed. It may be that with more time 
available students with manipulations in text would, indeed, have shown 
greater improvements in scores because of "time" spent with the materials. 
It is possible that one of the effects of providing students with 
instructional manipulations is to ensure their prolonged involvement in 
the task. It is also possible that no amount of inducement will keep 
students with "text only" reading that text, thus regardless of amount of 
time provided students who have text only may simply stop studying. 
Additional research in which shorter materials or longer time was 
available and "text only" students were provided with some external 
inducements such as instructions to "keep studying" might demonstrate 
this effect. 

Summary 

The relatively substantial support provided for hypothesis four was 
discussed and additional relevant issues considered. A number of issues 
relating to the larger problem of increasing learning efficiency were 


described and suggestions were made about additional research which may 


address these issues. 


Hypothesis Five 


Some differences will occur between performance scores of 
experimental and control groups on immediate and delayed 


testing. 


No a priori reasons exist for the expectation that the experimental 
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58 
manipulations themselves might produce differences in short and long term 
retention. The question asked is exploratory. Since not all of the 
students in the initial sample were available for delayed testing, only 
students for whom both test forms were available were used in these 
analyses. This necessitated analyses using an unequal n's approach 
(Winer, 1971). Table 10 presents the results of analyses of variances 
using Verbatim, Paraphrase and Total scores for the 102 students partici- 
pating in both immediate and delayed testing. 

These analyses show no significant effects due to groups but show a 
significant effect due to time of test in the analyses based on P and 
Total scores, F(1,91)=6.35, p¢.01; F(1,91)=6.81, p¢.01. This difference 
represents a significant decrease in P scores from immediate to delayed 
testing. Diminution of the more complex item scores after a one week 
delay seems consistent with the verbal learning literature which shows 


lower retention for more abstract or difficult material (cf. Hall, 1971). 


wee fl deen ‘sshd | 
itdeerg ae ar va 

) namie bipy 40 eBavisas 40% 
~busasq adishote ‘Sor pei: rm Ke 4 th ii bine yet m: 
Heed bale tabit eat 

hi were Jud Aawoty! 03 dub” eset Zeno bog has 4 
bem Yi cid) boesd seca a deat id sets ct 


joey oro 8 rade ib toea wt NeW! <a 


ewoda mhidw i Sali Fs sake tadiay og ae 
| 


-(i7eL) (ital a) ipsa bbe eam i 


i 
f be # 
a oe | 4 i} 


Scores 


Total 


Table 10 


Analyses of Variance with Repeated Measures of 


Verbatim, Paraphrase and Total Scores 
for all Experimental and Control Groups 


Source 


Between Subjects 

A (Group) Effects 

Subjects within 
Groups 

Within Subjects 

B (Test) Effects 

A x B Interaction 

B x Subjects within 
Groups 


Between Subjects 

A (Group) Effects 

Subjects within 
Groups 

Within Subjects 

B (Test) Effects 

A x B Interaction 

B x Subjects within 
Groups 


Between Subjects 

A (Group) Effects 

Subjects within 
Groups 

Within Subjects 

B (Test) Effects 

A x B Interaction 

B x Subjects within 
Groups 
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CHAPTER IV 


Conclusions and Implications 


In summary, hypothesis four, predicting superiority of experimental 
over control group scores received some support. Hypothesis one, pre- 
dicting the superiority of the graphic-word organizer over the prose 
organizer was reversed and the effects were statistically significant. 
Hypotheses two and three predicting differences between the two forms of 
objectives and the two arrangements of questions did not receive 
statistical support. The lack of statistical support obtained can be 
viewed as drawing attention to several important issues relevant both to 
further research and instructional practice. 

This research was conducted with the knowledge that experimental 
conditions are not conducive to optimal student motivation. It was 
assumed that in using three manipulations, all of which have previously 
demonstrated significant effects in experimental research, adverse 
motivational effects would not be so overwhelming. Post hoc analysis 
and examination of students' study materials now makes this assumption 
suspect. Kumar's (1975) discussion of evidence aie suggests that 
students were aware of the implications of "note' and "remember" comments 
yet they chose not to learn the material, implies a similar problem. 
Both this and Kumar's research have important implications for research 
using instructional variables. 

First, evidence that students have used the manipulations in the 
form of underlining, or answers to questions could provide an indicator 
of intent to learn. Similar to performance scores as evidence fOr 


learning, such indicators would naturally be inferential and indirect. 
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61 
It seems important to find mechanisms both to induce such decisions to 
learn and to provide evidence that they have been made. The use of 
students who enter the research situation with an expressed desire to 
learn information or skills should ensure initial motivational levels 
adequate to learning, while the provision of some external response 
should provide an indication that a decision has been made to learn the 
material under consideration. 

Second, it should be noted that the instructional contexts from 
which the revised forms of the manipulations and the hypotheses were 
drawn tend to be characterized by the use of such students, i.e., those 
who enter the situation with a desire to learn the material. Further, 
the materials are produced within a developmental-recursive system in 
which students are presented materials, respond to them and revisions are 
made on the basis of their responses. It may be a flaw in this research, 
common to other experimental research, that students were provided with 
materials but no special effort was made to ensure that they used the 
materials. The relatively high scores obtained by "manipulations only" 
control groups in respect to the experimental groups is evidence that 
it was possible to learn from them. Within this research it seems that 
while the manipulations may "provide support for the students' learning 
processes", they did not do so very effectively. This lack of effective- 
ness may be related to the lack of use rather than inadequacies of the 
manipulations themselves. 

In the literature reviews, it was pointed out that each of the three 
manipulations, organizers, objectives and questions, was associated with 
both supportive and non-supportive evidence with respect to improving 


learning from text. The inconclusiveness of the research was thought to 
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62 
be associated with various methodcelogical difficulties such as lack of 
operationalization, failure to control time and lack of consistency in 
presentation mode. 

In this research, it is suggested that the use of these manipulations 
in an experimental context may have obscured some of the critical features 
which may emerge when they are considered in an instructional context. 
While some information was obtained in this research relative to the 
effectiveness and ineffectiveness of the various manipulations, it was 
suggested that this information may lack external validity in terms of 
instructional uses. Thus, while the issues addressed by this research may 
be important to the production of instructional prose, not ensuring 
adequate initial motivation and not providing evidence of "intent to 
learn" may have resulted in an inadequate evaluation relative to the use 
of these manipulations in a "real life" context. 

It seems reasonable to conclude, from this research, that while all 
three manipulations may have a useful function in instructional prose, 
only the prose organizer made a significant contribution. This finding 
is of special interest because of its potential application to prose 
instruction. Among the many "rules of thumb" found in applied instruc- 
tional contexts is that which recommends interspersed visuals. In this 
case, the visual-verbal combination in the graphic-word organizer 
resulted in significantly less learning. While it is possible that this 
effect is a product of the particular experimental materials it is 
important that this issue receive additional attention, particularly in 
a more realistic context which ensures students' intent to learn. 

The data analyses demonstrating superiority of most experimental 


groups over control groups does allow the conclusion that when time is 
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63 
controlled, experimental groups will perform better than control groups. 
This conclusion should be considered with respect to relative "efficiency" 
of learning and also "efficiency" of materials development. While 
Carver's (1972) argument relating to "time" has been considered and does 
not receive support, the small magnitude of differences between experi- 
mental and control group scores should be noted. Such differences, 
even though significant, can not be used as compelling argument to support 
the preparations of organizers, objectives and questions for instructional 
prose. Indeed, the magnitude of the differences does not skew those 
obtained by Papay (1971) with single manipulations. This suggests that 
the assumption that all three manipulations are necessary in instruct- 
ional prose requires consideration. It is possible that any one of 
the manipulations alone is sufficient to provide support for the 
students’ learning processes and that including all three at most allows 
for "learner control" (Merril,. 1975) i.e., for the student to select the 
instructional support most appropriate to his/her own needs. This 
possibility is one which does not appear to have received direct attention 
in the literature and is one which may provide useful information. 

The major results of this research may be the attention drawn to 
students' entering motivation and intention towards the materials as a 
critical condition in attempting to investigate instructional variables 
within an experimental context. The suggestion drawn from these results 
is that while organizers, objectives and questions may be necessary 
"external events" to provide support for the students' learning processes, 
without appropriate entry motivational levels and external evidence of an 


intent to learn, they may not be sufficient conditions to promote 


learning. 
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It is felt that the attention to variables which influence internal 
validity (cf. Campbell & Stanley, 1963), such as novel materials, 
random assignment and experimental groupings may have severely limited 
the external validity of this research. Therefore, it is suggested that 
additional research involving these manipulations be conducted in two 
ways. First, research within a "real life" instructional context where 
the manipulations can be modified and improved until there is some reason 
to feel confident of their appropriateness to the particular learning 
materials should be conducted. Second, it is suggested that experimental 
research with these manipulations be conducted under less obtrusive 


conditions. 


De a _ PRA 
| 2 7, bf = a i> Me coaiaa 
oT ore Sahat Ry' rosa a -1aa, 


ie ° ( 
OF | 
oa < ioe 7 SF r nd & bo, 
: . 7 » 
: [ ria ’ a 
1 . 4 . io Dy a ian 
oi y : ; : 1 D a © oe 
aw Pusey ) 
Ani 
. 7 vA 


aoSe7 Sime Bh: Sanit’ sitiadi, | pel 


Bs 7 * Ra : A ai 7 ot ee. e 
ey spat “shai, gabe oe enoloe aa ew x ) 7 nae vt) 


We 
@iy Mi 


| | | one pee en ib: ee * 


a 
my A. yen ie Pire re a 
om , m i 


65 


References 


Ausubel, D. P. The psychology of meaningful verbal learning. New York: 


Grune & Stratton, 1963. 


Ausubel, D. P. Educational psychology; a cognitive view. New York: 


Holt, Rinehart & Winston, 1968. 


Ausubel, D. P. & Fitzgerald, D. Organizer, general background and 
antecedent learning variables in sequential verbal learning. Journal 


of Educational Psychology, 1962, 53, 243-249. 


Ammons, M. Objectives and outcomes. In Encyclopedia of Educational 
Research (4th ed.) New York: Macmillan Co., 1969. 


Anderson, R. C. Control of student mediating processes during verbal 
learning and instruction. Review of Educational Research, 1970, 40, 
349-369. 


Anderson, R. C. & Biddle, W. D. On asking people questions about what 


they are reading. In G. Bower (Ed.), Psychology of Learning and 
Motivation (Vol. 9) New York: Academic Press, 1975. 


Barnes, B. R. & Clawson, E. U. Do advance organizers facilitate learning? 
Recommendations for further research based on an analysis of studies. 
Review of Educational Research, 1975, 45, 637-659. 


Barron, R. & Cooper, R. Effects of advance organizers and grade level 
upon information acquisition from an instructional-level general 


science passage. National Reading Conference, 1972, 22, 78-82. 


Barron, R. F. & Stone, V. F. The effect of student constructed graphic 
post organizers upon learning vocabulary relationships. National 


Reading Conference, 1974, 23, 172-175. 
Bassett, -R. E. & Kibler, R. Su :kEfect of training in the.use of 


behavioral objectives on student achievement. Journal of Experimental 
Education, 1975, 44, 12 - 16. 


Blanton, B. E. Reception learning and advanced organizers: implications 


for reading research. National Reading Conference, 1971, 21, 3-7. 


Bloom, B. S., Hastings, J. T., & Madaus, G. F. Handbook on formative and 


summative evaluation of student learning. New York: McGraw-Hill, 19/1. 


Bransford, J. D. & Johnson, M. K. Contextual prerequisites for under- 
standing: some investigations of comprehension and recall. Journal 


of Verbal Learning and Verbal Behavior, 19924 daipes7) 82.6% 
Campbell, D. & Stanley, J. Experimental and quasi-experimental designs 


for research. Chicago: Rand McNally, 1963. 


Wat mar Poca ; ee Le tom li 
\ : ; W ro 4 +, 

“0h | on ae De aay is 
hon i 


bre bes voablaed ts ane 
fanegot) saninxbsd me) : 


Sano kta vb7 ie re 
oF ae 
RAR! 


some : a 


: Bgtboc sista ” to 
eee ivak , 


lecray SAITO \2 associ r 
, 08 Pas Xi Ps » dinsadan 2 Bi : 


bude is yank at be) aiied «AC 
TLe 7 ae oa, hevA sidieT wot. | 


fgniatset ats9kifos3 +: wy Ssodevbhe ° 
-eStbasa ‘is 2 9 oh ae oh ad a bi 
$C28-TCR te CNeh? 


isyal sbextgi bes arosinags) goonebe 3 
le einai iq¥ai~leeorIoireat! f 
6-4) | ,.2S' Stel: SIs 3 FIGS) 


Sitigeta betsy tteras tnebuds to re 
SeROLIGM » eK ittzuctisls 


Re Hou edt si gnigtedd/ 26 3>e23T . L.A) 
letsomt Asmuol .itsmsyetiioe Jaobuse fo a 
. at a Si _ 


stotieotiqmh “Parssioags« bepaeyis, bua) an bt oss OKT eo BF a .6 _nossels 
VE gS QANCi .9.019753NG9 Bitheex [snohiAn) Bomebeat galeaba aed 


Maw soodbnrgl wD yevabslh 6 ..7 10 p,aideal . eG) moods 
Bret TOY wal, sgmtniest Jnobutg 2g) 


STSbn i167 sejtetuporeiag Teetxoine) » orndel o .0 AL .dvoteae7d 
senteol i feces bas noienedsagico 206 09 Amos ee 


MSVNIN (2T (SNCR Veokveded [ids 


a 


aeeurs 


sia base Logan 


- 


ae tash Lernemi zs yx; bie f: dpa bteols i) free b 4 ‘aeons 


66 


Carey, S. T. & Lockhart, R. S. Encoding differences in recognition and 


recall. Memory and Cognition, 1973, 1, 297-300. 


Carver, R. P. A critical review of mathamagenic behaviors and the effect 
of questions upon the retention of prose materials. Journal of Reading 
Behavior, 1972, 4, 93-119. 


Corrozi, J. F. The effects of reading time, type of question and 
instructional format on short and long term retention of relevant and 
incidental prose materials. (Doctoral dissertation, University of 
Delaware, 1971) Dissertation Abstracts International, #71-6, 465. 


Courseware Inc. The Systems Approach, a manual produced by Courseware 
Me. Pe OF Box G1], Provo. Utah, 19 74:. 


Craik, F. I. M. & Lockhart, R. S. Levels of processing: a framework for 


memory research. Journal of Verbal Learning and Verbal Behavior, 
P72 11, 671-684. 


DeCecco, J. P. & Crawford, W. R. The psychology of learning and 


instruction. New Jersey: Prentice-Hall, 1974. 


Dooling, D. J. & Lachman, R. Effects of comprehension on retention of 


prose. Journal of Experimental Psychology, 1971, 88, 216-222. 


Dooling, D..J* & Muddet oR. L.» Locus of thematic. effects in retention 


Of prose. Journal of ExperimentaliPsychology, 1973, 90, 272-278. 


Duchastel, P. C. & Merrill, P. F. The effects of objectives on learning: 
a review of empirical studies. Review of Educational Research, 1973, 


43, 53-69. 


Etter, D. C. Adult learner characteristics and instructional objectives. 
(Doctoral dissertation, University of California, 1969) Dissertation 
Abstracts International, #70-8140. 


Frase, L. T. Learning from prose material: length of passage, knowledge 
of results and position of questions. Journal of Educational 


Psychology, 1967, 58, 266-272. 


Frase, L. T. Effects of question location, pacing and mode of retention 


of prose material. Journal of Educational Psychology, 1968, 59, 
244-249. 


Frase, L. F. Boundary conditions for mathamagenic behaviors. Review of 
Educational Research, 1970, 40, 337-348. 


Frase, L. T. Effects of incentive variables and type of adjunct questions 


upon text learning. Journal of Educational Psychology, 1971, ‘G2, 
Dipl — Si D = 


Frase, L. T., Patrick, Z. M. & Schumer, H. Effect of question position 
and frequency on learning from text under different levels of incentive. 


Journal of Educational Psychology, 1970, 61,- 592-56. 


in - 


sts) 


inkvs 843 hie. 4 ck Pastel we 
ginthess 30 | tay thot, patel iy tia A, LoWnok Ine wR 4 


| wv y ei 


fos noblas.p Bo Snag 8 gothass Yo are 

bit’ Unavel2: To mo. nsies ued Baot hoe sande pita 
te vohetov inl. , babis 1teee tb Brot 2ot8) sleds 
(2a) GH ETE. Gao bier ae sdpeisedé ot ots ad 4 : 


yy ve. 
stewoatva) Vd ‘bests 5 ,doworqnys segcaee 


A PVCL Gries ove 


Yor. Frowomiga « tynteesdogy ste Bhaved oe + Aa ae d 
Ioivsrss. incre!) bus Qrenyied Ledtet 0 Aide 


bho snitch) So veo koee ae SAT oS oH -bianwerD 
Ser ‘ay ta ptt al ese ial wel 


10 996i JRa7e7, nO heat: vetoes toe Sebel aa Seals toe 
SSSHOLY p BE A 188 peat wo. Anaieonyny us 2M 

, . a hy 
cohtwsset ak sasotits eres th édoait ad vi he hit 


pBRSTENS AGE, ENOL «CAG teene eae ea 


Sikhs £46 ‘pov hin: pie: fol e3a574B Pet st ia “pilty 
ETRE domaine” od Lisbb bs edt Te ye ome gaprhlts one 


+aBvisgetde Lunot tourtait bow -eoliatyeioe van: pr hab 
nolsedisaghtt (SOs ,atrtéiplas kb yodesovtay ya , Rh teeoge 
| 1818-008 5 Sane gatos ade 


egnad woot saRberrc to isgoat. BE Voor eegera wats pitaened'. 
DagiOc3sp ht Yo Lopsegt ©. enol yasvp) 34, aok ry Ome les 
. NEARS 88 a eyaok 
* | 2 oie ca 


7! 


foldneded To’ lk om Bra antge 
eRE: BORE , eposo 


Paguis t Leni Paden ap oe “s 
ia A 


to worish Bavkvailod errr Sa cut hides) \cteliow er of Da Ps 


Pe) 
iA Seah t6E .Oh OTRE Sigiegt A ano! 


— 


-s ie 05 ° 
ei s 
¥ 


i ’ ay: i’ 
era bt}asin tunttbs to Sqyd nk S452 re ev Pi ggant ibs 
ead cEKSE “goto a Do eas 


meee tebe 16, rere ponte Bs 
sovitsiest 10 elsve! J0s- 168th ue thus wort 
' 80-88", Bho! NAOT 


~ 


oHag 


67 


Gagne, R. M. Educational technology and the learning process. Educational 
Technology, January, 1974, 3-8. 


Geis, G. L. Behavioral objectives: a selected bibliography and brief 


review. Stanford University: California. (Eric Clearinghouse on 
educational media and technology, 1972). 


Geis, G. L. & Chapman, R. Knowledge of results and other possible 
reinforcers in self-instructional systems. Educational Technology, 
April, 1971, 38-49. 


Hall, J. F. Verbal learning and retention. New York: J. B. Lippincott 
Coe, L974" 


Hartley, J. & Davies, I. K. Preinstructional Strategies: the role of 
pretests, behavioral objectives, overviews and advance organizers. 
Review of Educational Research, 1976, 46, 239-265. 


Hiller, J. H. Learning from prose text. Journal of Educational 
Psychology, 1974, 66, 202-211. 


Holliday, W. G. Teaching verbal chains using flow diagrams and texts. 
AV_ Communications Review, 1976, 24, 63-78. 


Ingersoll, G. & DiVesta, F. Effects of modality preferences on perform- 
ance on a bisensory missing-units task. Journal of Experimental 
Esycholorey,21972,2 93% (no. 2) 5, p.2 386-391. 


Kaplan, R. & Rothkopf, E. Z. Instructional objectives as directions to 


learners. Journal of Educational Psychology, 1974, 66, 448-457. 


Kumar, V. K. Functional significance of orienters in prose learning. 
(Doctoral dissertation, University of Wisconsin, 1972) Dissertation 
Abstracts International #72-15, 364. 


Kumar, V. K. Functional significance of orienters in prose learning. 


Journal of Experimental Education, 1975, 43, 72-76. 


Kurtz, P. D. The effects of practice and knowledge of instructional 
objectives on learning from prose. (Doctoral dissertation, University 
of Michigan, 1972) Dissertation Abstracts International #72-29, 125. 


Ladas, H. The mathamagenic effects of factual review questions on the 
learning of incidental information: a critical review. Review of 
Educational Research, 1973, 43, 71-82. 


Lawson, T. E. Effects of instructional objectives on learning and 
retention. Instructional Science, 1974, 3, 1-22. 


Mager, R. F. Preparing instructional objectives. Palo Alto, California: 
Fearon, 1962. 


ae 


Lptips aon Suh ascot vin ba wen 


sy 
wn 


Joiyd baths wins bi ldtd: bavaral cen hs aes : oa : a 
HO 98 Tonga TEeLO » Ma) | etera eh au F 
ee ‘eselnda a 


ny, Has 

‘ 

: adhesion 
aon ehtihsabie yo. 


oi 0teegq Mehdi ‘bas eg foe 
. Keo londaal (agrot csAoU ba: 


Frosniqnhtl 8... ama F 4 Be 1 meg ery sad Liss 
" a . f ! 


ye ue 


io alot Sct ies kaesanae tas Sityan fers a elu 

sVTss (negro. shiavbs bes a dl Lav saubtestee ce: 
.@al-Ged Vie ole! domes 7 righ: 

| iol . Ba 

eho Rm ke rd pa = 

risweds: Tee VOD oS" ne Ww 

| * ye 

tee" bie embigcih ~oit eae aes) Ledvew aahdanet | 2 ww ch 

} ua & 8S ,aRRI “3 en 


Tiotwey wovessaits tote riled qn B3 2 th pen’ 
Le irsm@lisagsa io Lsiteot Mc's oti ay Sib ties : 
si aac ose .4° tS lee) ce a 


vat thee 
or a. 


of asokroawit $4 hi hee avis aa ive? fsangt 4U seit I/ ier te lee | 
Ch BWN 8d LANES ehgokg fory2 6 : isa eube Jo. 


sotnxbel ‘sa04 ty eTSc09 i790: th sma hitagie 
. = > a\ 3 P ee is ao Ds 
fpigaiagaeie (iter entero i a ie ah ani Xn 
Sues | aie ge 


-gmirribse saotq. at 3 re dhiales 14 Sear tagta Lede syani’s 2 


ce | eR ever OSM Subs | 


| 

lefokemfizea! 34 Rerey fo: bh soltoa+q 20 ateetis at? (Ag + ot 

(iiersvin. nai yayrsae th rend .o8er¢ or? ghkiteel no seviz oS) | 
wed REASNN EOS Aare, 3 este sonia senate tenes epagte vi ties i! 


© gA rt rat 

oid > ene) taaup wolvsy mesa? 14 2309etts s aieidadiiaa oft! LH (eabed : 

20 WaEVSA * .HOTVS7 fostae rd J narso dst see sage ai pitt hepa: ia 
ee ie Ms GA-iv cS 200 (ibweeee fap be: 


bas antotost nb aavi igs cit fenotigatsagt ta etnekTy 


pete “$n ah Mae Pare toe. line \aoosaget eis 

} ; ¢ 

a Pat . ae | (eg ih 
ainMOTRIRA OIL of ef Wideetis. Saisots.9130ur pale 92) 


68 


Merrill, M. D. Learner control: beyond aptitude-treatment-interactions. 
A-V_ Communications Review, 1975, i2oy IZ 1=226. 


Merril), M.D. .&Boutwell; R. C. Instructional development: methodology 
and research. In F. N. Kerlinger (Ed.) Review of research in 
education. Itasca, Illinois: F. E. Peacock, 1973. 


Merrigiy .PoeFs» Interactiontof cognitive abilities with availability of 
behavioral objective in learning a hierarchical task by CAI. 


Technical Report #5, Austin, Texas: CAI-Lab, University of Texas, 1970. 


Merrill, &. F. & Towle, N..J. UInteraction of abilities and anxiety with 
availability of objectives and/or test items on computer-based task 


performances. Proceedings of the 79th Annual Convention of the 
American Psychological Association, 1971, 539-540. 


Mistler-Lachman, J. L. Depth of comprehension and sentence memory. 


Journal of Verbal Learning and Verbal Behavior, 1974, 13, 98-106. 


Morasky, R. L. & Willcox, H. H. Time required to process information as 
a function of question placement. American Educational Research 
Journal, 1970.7, 561-567, 

Morse, J. & Tillman, M. Effects on achievement of possession of 
behavioral objectives and training concerning their use. Paper 


presented at the annual convention of the American Educational 


Research Association, Chicago: 1972. 


Papay, J. An investigation of the effects of type, location and 
distribution of orienting stimuli on the acquisition and retention of 
meaningful prose materials. (Doctoral dissertation, University of 
Florida, 1971) Dissertation Abstracts International, #72-277. 


Peeck, J. The effect of prequestions on delayed retention of prose 


material. Journal of Educational Psychology, 1970, 61, 241-246. 


Popham, W. J. Evaluating instruction. Englewood Cliffs, New Jersey: 
Prentice-Hall, Inc., 1973. 


Popham, W. J. & Baker, E. L. Establishing instructional goals. 
Englewood Cliffs, New Jersey: Prentice-Hall Inc., 1970. 


Rickards, J. & DiVesta, F. Type and frequency of questions in process- 


ing textual material. Journal of Educational Psychology, 1974, 66, 
354-362. 


Rothkopf, E. Z. Some theoretical and experimental approaches to problems 


in written instructions. In J. D. Krumboltz (Ed.), Learning and the 
Educational Process. Chicago: Rand McNally, 1965. 


Rothkopf, E. Z. Learning from written instructive materials: an 
exploration of the control of inspection behaviors by test-like 


events. American Educational Research Journal, 1966, 5, 144-145. 


py 


es 


ae ya seed ah ai Cb 
ope tia | 


20 ie AS tg. clit to ee 
« TRE ates Tage 
OC! -eseet) ta VileeevN 4 4 ; 


tite yrphiies) brs eskanrideg, 
fond bo@antesunio® TO: 
917. 2G NOt tus vey 


Pisin sIbS20oe bak te: 
ei BE ak ewes . 2obys 


ae ncobderna’ 245070 463 beet lip? 3 


ot a iat Sth 


tH R TOL RE92B 50 qo Ber am ays Lik 1G ani? 
72 ed Srey) 25 aio 1 Bina tea 
Sas; td Meaubll fie co fens 4 ae ip ae 


late, nays SWS Vie oxaaRie BP. ‘Ye 


te nokiasgea fis Lok idguiat yi s 
lo ySlesSyi RU oolhistyscoth Ie 
VY. GREE Limoksaats at ed 


ey i ito Notsucsyss fo vale ne sible | 


Oi-Tek fo OTe! . vee 


‘ea au wast) tommy’ 


oe 
md “t 


sarap tg mi enoiseoup to gedowney baé oqyT: .F vee ab .o putamen 
MHS 6 Soria: Saale eee, - 


8 ee ¥ 


dine N6ody. wy, Gedseotges Sesndebeaes bac tao: revo painge ‘2 Be 
Sit Sie piketeet .{.b%) ealowers.¢ tol ee souetani madi 


To od saat i 
 \icbabeien Sweka tas 


| 4 Og econ avis 
| ane oi: 2o -zgath 


' Hie ye) ‘see a t piers 
ees RL A how oo RA TSY Te 


JGeitokik enact cupsa omtT oH ca 
igapaasy! Le feed R+s 14 yt ne LS co ramegeig. t ce 


"0 ele ttetagt seer Lttantasam) 
fetes al “ie Ns oy 3 i . 


Rgntiaiivess’ .1 .ad eid ®. 1 Hi sampeigot oy 
q 


; a6 SDs: Ae 


yet 
ne ie 


ae ay me ae : | 


‘ 1 we al | 


? po eee ee 


uothaghaesiak| ak | 2: sangeet 
ri) 4p gtr’ Jace ited aeeh | 


oe 


> ae J 


6 suas ost iti yalowe | 
invade fs 


‘hy ~ # qf: a. ‘< 
rane om 64-noP$net9 


iting | 


(oeleh) VpM pee ho bootes Lgard oh 


i ' i 
f 


bret .-nye eae Res as 


naa tty nee fal 8.3 ,igediizon mh, 
. Py 


ag3 Toe al Piniativecant to loxtHoges oe 
Fas yok tornsas! Lavo l eae vat soe 


Rothkopf, E. Z. The concept of mathamagenic activities. Review of 
Educational Research, 1970, 40, 325-336. 


Rothkopf, E. Z. & Bibicos, B. E. Selective facilitation effects of 
interspersed questions on learning from written materials. Journal 


of Educational Psychology, 1967, 58; 56-61 


Rothkopf, E. Z. & Billington, M. J. Relevance and similarity of text 
elements to descriptions of learning goals. Paper presented at the 


annual meeting of the American Educational Research Association, 
Washington, D. C.: 1975. 


Rothkopf, E. Z. & Kaplan, R. Explaining the effect of density and 
specificity of instructional objectives on learning from text. 


Journal of Educational Psychology, 1972, 63, 295-302. 


Salomon, G. What is learned and how it is taught: The interaction 
between media, message, task and learner. In D. R. Olson (Ed.), 
Media and symbols: the form of expression, communication and 
education. 73rd Yearbook of the National Society for Studies in 
Education, Chicago: University of Chicago Press, 1974, 383-406. 


Smith, J. M. Relations among behavioral objectives, time of acquisition 
and retention. (Doctoral dissertation, University of Maryland, 1970) 
Dissertation Abstracts International, #71-4526. 


Swenson, I. & Kulhavy, R. W. Adjunct questions and the comprehension of 


prose by children. Journal of Educational Psychology, 1974, 66, 
242"215; 


Tobias, S. Review of the response mode issue. Review of Educational 
Research, 1973, 43, 193-204. 


Wilson, S. Tis Koran, J< Also -ovhoran, Mo eb. ebitfenenrral responsesto 
inserted questions when learning from written materials containing 


two modes of instructional content. Paper presented at the annual 


meeting of the American Educational Research Association, Chicago: 
1974. 


Winer, B. J. Statistical principles in experimental design. New York: 
McGraw-Hill, 1971. 


Wittrock, M. C. Learning as a generative process. Educational 


Psychologist, 1974, AAS S299 


69 


otibetupbs Fo Smit pas hoe rae Pexiy: SAG pode 8 aaa , m oe 
er san of le uM +6 Vii Ve ob P GF yd th iste NTH NPaeda any vane md, hrs 


Lg 


notanedsaqaOy ott hos andbiagay dint hhe Wt valle a 5 oe 
wo OS Ribs +MaQod gEYS a piapauy =e 2 ba aes sna \ a wig 
j | : . t We sag | ei ( 
yo A ; 7 j | ahi ; ' ‘ ari * 
Landigssubh So waived psuauk ebett weer a6) An walvs 
peel ne 


: ‘. .{ ‘ vy ue iG , : 
“Og manana lal tives 32420" ipae if F m % pues ‘oy a ‘ 


a | if a hs , iH a) arene) ik 6 
: : ¥ : A | w Vile Pas 

} } ” i } ‘t f 7 5. t 
Ade i ee > 


ii LM 
» 7 t 
Sa IAL YT Lo 
é i het mo iY fi 
a ae 
j an 
foals eft ~eordpvisss ki gbmagsl deut Xp 
* at ie a 7 ah faa r Ae ; ¥ é] 
, ( MARERESE MBE 2008 


‘nd ay ee me 


~ a = “ * ve a i" © . “7 
16 arosits ootiest ay euas: bbe eh es a tae eee oe 
a ‘ ~~ ¥ Oye - nee »~ 
Wiuck aleleeeent x eee he a padtisasd ce: 1 Pie ich yoge 73. 


7 ’ fs Pr ; ’ > ‘ p 4 9 

Lede. eG wi ky 29) : wtipoubs 2¢ 
A _— iat . om ia serene ria kD 

} ie : : oh » j 


*xS7 to v*kupi bic ocr RSTOE IM a eee 
ey bey ds Jb tecety, ds, miayesi L By 
. et a BS eel rrerignd , r | ar. 
ie tsasesr Pe eter tI {aad Tah Aso IL wee hay 


~— ee <email Lhahable eee eee meee a 


- deal 

; Ea ar baie 
er eos ‘ 
hae. vt keueb to aoe the Si TRAE fexa ow HG pee é. +o a et 
4483 won?t gakcteteel Bavitesfpda, Letens aaa. 9 ie ‘ethan tay hasge 
coe s hi gy SPOl Vigodor ‘a4 haghaeoubs ho [apie 
ee a ot. a ie a 7 r —7 - 

iy / ae . 

noltigtedat sit.  eaueRiab 2% wed foe Te saxaet ad +i ee 

Poa noe Q off re \eROnTesd. bil tas7 . 23.0 89Ar ue be pave eb Te 


bree ‘ro  touatios .geteessuxs 2o mot acy” Ap pang 4 nthor 
2 ui V0? Sidfover tena fiat u rey a0 ter Anogkages ‘Bae na ibe 
\POSSERE, VPLED [eRsat eesan® zo (hic say qu beat Py: aaeres Sa 


M ” * a. 4 


va xe 


. Love pat ‘ey, Lage orn via 2d abst! _ipbasyn - 
| oa “aa 5 a 


a 
y 


ALERT Gh uti L it rr 2208 
\ wy ee se ; Sak ‘i 


grotheatogsa ala beter na Sih ata ne raed) 0s meohseoup. aig 
ee ‘tts 16° ¢_boiobe ge aah -- deans Lnad eae yaak to genes 


> Osh eoxno age z pie abenA t BOSS Lrg jijeiuh? geek Ha, pi 3 ‘26 palde o 
7 iy ” > % a _ a8 he, 
( al 


u ‘7 
7 + 
: 


7 : oy 5 
44 : Pee ga Fx 
:020Y wet) -optaap, Led mem eaegas al sola? oaksé 1072 .l -@ preckW 
A ; : os at: a anaraammael as 


%y ; Ne | 
¥ Az uo 

bes Ih F "85 Stexsnag & "2 
i. «tR> y 


APPENDIX I 


List of Experimental and Control Groups 


70 


a: } “ re -; 7 b, 
‘ib ty ae VY i 
i] ? Whit A i, yy 


¥ 


l => 1 pp Ore- ; 
ry? aN were, 
( AT .| 
. ae 0 es 
vt wa i : 
a it 4 4 bli, 
( vs ty ‘i Revie 
‘ j Me | 
it } ef i ri 
ia ae 4 
\ 


ip 


Experimental Groups 


Las 


Prose organizer, inconsistent questions, content-referenced 
objectives and text (111). 


Prose organizer, inconsistent questions, content-free objectives 
and text (CLi2). 


Prose organizer, consistent questions, content-referenced 
objectives and text (121). 


Prose organizer, consistent questions, content-free objectives and 
text (122). 


Graphic-word organizer, inconsistent questions, content-referenced 
objectives and text (211). 


Graphic-word organizer, inconsistent questions, content-free 
objectives and text (212). 


Graphic-word organizer, consistent questions, content-referenced 
objectives and text (221). 


Graphic-word organizer, consistent questions, content-free 
objectives and text (222). 


Control Groups 


9. 


LO; 


a 


Prose organizer, inconsistent questions, content-referenced 
objectives with no text. 


Graphic-word organizer, consistent questions, content-free 
objectives with no text. 


Text only with no organizer, objectives or questions. 
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Section 1 


The Endocrinology of Pubescence 


Pubescence refers to a characteristic group of bodily changes 
associated with sexual maturation. These bodily changes do not begin 
simultaneously but tend, rather, to follow each other in a definite order 
which is identical for all persons in a given sex group. This sequence 
is the same irrespective of the age at which pubescence occurs--even in 
instances of pathologically early or pathologically late puberty. It is 
also similar in all races, cultures and climates. 

Just as long as the environment is sufficiently favorable to 
sustain life during the adolescent years, the bodily changes of 
pubescence invariably take place in the same order. In boys, for example, 
the first pubescent change is growth of the testes, followed by the 
appearance of pubic hair, by voice changes, and by the full development 
of facial hair. Enlargement of the breasts is the first indication of 
female pubescence. It is followed by pubic hair and the first 
menstruation. 


Section 2 


Initiation of Pubescence 
Prepubescent boys and girls are not asexual biochemically. Sex 

hormones are found in the blood stream during childhood, but in insuf- 
ficient quantities to result in primary and secondary sex characteristic-. 
These hormones are produced by the adrenal cortex under the stimulating 
influence of the corticotropic hormone of the anterior pituitary gland. 
The adrenal cortex produces both male (androgenic) and female (estrogenic) 
hormones, about the same quantity of each in both sexes. Hence there is 


no significant difference between prepubescent boys and girls in 
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74 
androgenic-estrogenic ratio. 

Pubescence is initiated by several important endocrinological changes. 
First, the concentration of the sex hormones (both androgens and 
estrogens) rises markedly and stimulates the development of primary and 
secondary sex characteristics. This change reflects in part the 
increased stimulation of the adrenal cortex by the corticotropic hormone. 
In each sex, therefore, the adrenal cortex produces both more androgens 
and more estrogens during pubescence than during childhood. Even more 
important, however, in raising the level of sex hormones is the fact 
that the gonads (testes in male, ovaries in female) begin secreting sex 
hormones for the first time. Such secretion first becomes possible 
during pubescence because the gonad-stimulating hormones of the anterior 
pituitary gland (the gonadotropic hormones) first become functional and 
stimulate gonadal secretion at this time. 

Second, since the testes only secrete androgens and the ovaries 
only secrete estrogens, pubescent boys and girls are no longer undiffer- 
entiated with respect to the kind of sex hormone they produce. Boys 
produce more androgens than estrogens, and girls produce more estrogens 
than androgens. Still a third important factor promoting the initiation 
of pubescence is an increase in tissue responsiveness to hormonal 
stimulation. The various tissues involved in the primary and secondary 
sex characteristics become more responsive than previously to gonadal 


stimulation, and the gonads themselves become more responsive to 


gonadotropic stimulation. 
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Section 3 


Primary and Secondary Sex Characteristics 


One major group of pubescent changes occurs only within a given 
sex. These are the primary sex characteristics, and involve both the 
internal and external organs of reproduction and copulation. The 
second major category of pubescent development (i.e., the secondary sex 
characteristics) occurs in both sexes. Some of the latter, such as the 
growth of axillary and pubic hair and the increased activity of the , 
sweat glands, are practically identical in the two sexes. Other secondary 
sex characteristics, however, such as breast development, facial hair, 
deepening of the voice, and the attributes of male and female body form 
are much more pronounced in one sex than in the other. 

Primary and secondary sex characteristics manifest both normal 
and pathological degrees of variability. Only one type of variability is 
possible with regard to the primary sex characteristics: An individual 
may be more or less well-developed with respect to those organs character- 
istic of his sex group. Secondary sex characteristics, however, vary 
on two separate dimensions: First, an individual may be more or less 
well-developed with respect to sex-appropriate secondary sex character- 
istics (e.g., breasts in female; facial hair in male). Second, he may 
also be more or less well-developed with respect to sex-inappropriate 


secondary sex characteristics (e.g., facial hair in female; breasts in 


male). 


Section 4 


Normal Overlapping Between the Sexes in Sex Characteristics 


It is apparent, therefore, that varying degrees of overlapping 


ordinarily prevail between the sexes with respect to the different sex 
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76 
characteristics. Secondary sex characteristics are, by definition, found 
in both sexes. In those instances where they are undifferentiated with 
respect to sex (e.g., axillary hair), the overlapping is complete: no 
differences between the two sexes exist. In other instances, however, 
where secondary sex characteristics are normally more pronounced in one 
sex group than in the other, overlapping only occurs at the extremes of 
the distribution, i.e., where variation is extreme enough to be consid- 
ered pathological. This situation is illustrated when a given sex- 
appropriate characteristic (e.g., breasts in female) happens to be both 
poorly developed in one sex (female, in this example) and, simultaneously, 
pathologically well-developed or sex-inappropriate in the opposite sex 
(male, in this example). 

Primary sex characteristics, on the other hand, are, by definition, 
as well as in fact, distinctive of just a single sex group. Hermaphro- 
ditism, i.e., the occurrence of both male and female primary sex organs 
in one person, is an extremely rare genic accident. 

Section 5 

For all practical purposes, therefore, the development of either 
male or female primary sex organs in a given individual is one of the 
uniform or invariable attributes of pubescence. Individual differences 
in primary sex characteristics conform to the bell-shaped distribution 
curve--but only within a given sex, not between the two sexes. This type 
of "either-or" distribution of primary sex characteristics (i.e., the 
existence of two entirely separate, non-overlapping distributions of 
individual differences) is called "dichotomous." It contrasts sharply 


with the continuous, overlapping type of distribution found for the 
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7? 
secondary sex characteristics (i.e., the occurrence of individual 
differences between the sexes as well as within each sex group). 

As pointed out above, depending on whether or not a particular 
secondary sex characteristic is or is not more distinctive of one sex 
than another (e.g., breast development as compared to axillary hair), 
overlapping will either occur in only extreme cases, or will be complete. 
By "overlapping" in this discussion, we mean overlapping degree of 
development of a given characteristic that crosses sex boundaries, and 
not simply the fact that all secondary sex characteristics are, more or 
less, represented in both sexes. Overlapping, in other words, is not 
exemplified by the fact that all males have rudimentary breasts and all 
females have some facial hair, but rather by the fact that some males 
have larger breasts than some females, and some females have more facial 
hair than some males. 


Section 6 


Hormonal Regulation of Primary 
and Secondary Sex Characteristics 


Male and female sex hormones have either a selective or an 

unselective effect on the development of sex characteristics. They do 

not neutralize each other chemically. In the case of the primary sex 
organs and the sex-appropriate secondary sex characteristics, the action 
of sex hormones is selective. Male sex hormones, (androgens) for example, 
stimulate the growth of the testes anc of facial hair, and tend to inhibit 
the growth of the female primary sex organs (e.g., uterus) and of female 
sex-appropriate secondary characteristics, and inhibit corresponding male 
characteristics. Thus, since the pubescent female normally produces more 


estrogens than androgens, her feminine sex characteristics (e.g., breasts, 


female body form) are more highly developed than her masculine sex 
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characteristics (e.g., facial hair, male body form). The opposite 
situation obviously prevails in males. In the case of such undifferenti- 
ated secondary sex characteristics as the growth of axillary and pubic 
hair, the action of estrogens and androgens is unselective: male and 
female sex hormones have identical stimulating effects. 

Differential hormonal effects, however, cannot explain the 
uniformity or invariability in the order in which the various primary 
and secondary sex characteristics emerge in a given sex group during 
pubescence. The same concentrations of the same sex hormones are equally 
accessible to all bodily itssues, and would therefore stimulate simultan- 
eous development of all sex characteristics if the factor of differential 
tissue responsiveness were not operative. Hence, it is reasonable to 
assume that some tissues become responsive to the stimulating action of 
sex hormones earlier than do others, and that the order in which this 
responsiveness is acquired tends to be fixed or invariable within each 
sex group. 
Section 7 


Pathological Variations in Pubescent Development 
Age of Pubescence 


Pubescence normally takes place between the ages of ten and 
seventeen, with a mean of approximately 13.0 years in girls and of 14.5 
years in boys. Within the normal range of variability, the age of 
pubescence reflects the age at which gonadotropic and gonadal hormones 
are secreted in functional quantities, as well as the age at which 
increased tissue responsiveness to the stimulating action of these 
hormones is acquired. Individual differences in these latter phenomena, 


in turn, are determined by hereditary differences (and hence are related 
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to age of pubescence in parents and siblings) and also by such environ- 
mental factors as climate, nutrition, and general health. Pubescence, 
for example, generally occurs earlier in temperate than in tropical or 
Arctic climates; and the earlier average age of pubescence in recent 
decades is largely attributable to improved standards of nutrition and 
general health. 
Section 8 
Precocious Puberty 

Pathologically early (precocious) puberty, on the other hand, is 
almost always caused by excessive secretion of the adrenal cortex in 
young children or, more indirectly, by excessive secretion of cortico- 
tropic hormone by the anterior pituitary gland, and hence overstimulation 
of the adrenal cortex. The underlying pathology is an overgrowth (hyper- 
trophy) of gland itssue in either the adrenal cortex or anterior 
pituitary glands. 

The kinds and sequence of pubescent changes in precocious 
puberty are identical in every respect to those occurring in normal 
puberty. The only difference is that they occur prematurely. The con- 
dition is treated most effectively by surgical removal of. part of the 
hypertrophic gland involved (i.e. , adrenal cortex or anterior 
pituitary). 


Section 9 


Delayed Puberty 
Pathologically delayed puberty generally reflects gross (extreme) 


insufficiency of gonadal production. This undersecretion may be caused 
by such destructive lesions of the ovaries or testes as bilateral cases 


of mumps, tuberculosis, and cancer involving these organs. 
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Destructive (atrophic) lesions of the anterior pituitary gland have 
the same retarding effect on the onset of pubescence as destructive 
lesions of the gonads. Under these conditions, insufficient gonadotropic 
hormones are secreted, and the inadequately stimulated gonads accordingly 
fail to produce adequate quantities of sex hormones. The occurrence of 
pubescence may also be abnormally delayed in instances of prolonged and 
severe malnutrition (at a near-starvation level), and in instances of 
extreme and protracted emotional trauma. Such cases are rarely observed 
under ordinary circumstances, but were not uncommonly encountered among 
teenage inmates of the concentration camps in Nazi Germany. 

Pathologically delayed pubescence requires attention to the 
underlying cause. This may involve medical or surgical treatment of the 
gonadal or anterior pituitary lesion, on the one hand, or correction of 
malnutrition and/or emotional trauma, on the other. In any case, however, 


injections of large doses of appropriate sex hormone are of appreciable 


value. 


Section 10 


Pathological Variability in Primary 
and Secondary Sex Characteristics 


Underdevelopment is the only kind of variability with respect to 
the primary sex organs which is medically and behaviorally significant. 
Where the cause is hormonal in origin, there is naturally a history of 
pathologically delayed pubescence, as well as generalized signs of 
inadequately developed primary and sex-appropriate secondary sex character- 
istics. Gonadal insufficiency in a male teenager, for example, would be 
manifested by undersized penis and testes, lack of body and facial hair, 
high-pitched voice, etc. 


A much more common cause of underdeveloped primary sex organs, 
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however, is localized tissue under-responsiveness to normal hormonal 
stimulation. The gonads in this instance secrete normal amounts of sex 
hormones which induce normal development of all primary and secondary 

sex characteristics with the sole exception of the one hormonally under- 
responsive primary sex organ in question. The abnormality, therefore, is 
strictly localized to this one organ. 

Section 11 

Secondary Sex Characteristics 

Pathological variability affecting the secondary sex characteristics 
is somewhat more complicated. Localized tissue under-responsiveness is 
again the most common cause of underdeveloped sex-appropriate secondary 
sex characteristics, giving rise, for example, to such isolated abnormali- 
ties as high-pitched voice in males and poorly developed breasts in 
females. In addition, however, localized over-responsiveness to opposite 
sex hormones is also a cause of significant abnormality insofar as the 
secondary sex characteristics are concerned. If the facial hair of a 
female happens to be over-responsive to androgens, she will grow a minia- 
ture beard. Similarly, if the breasts of a male happen to be over-respon- 
sive to estrogens, they will be unduly enlarged. 

In both tissue under-responsiveness to sex-appropriate sex hormones 
and tissue over-responsiveness to opposite-sex hormones, the pathological 
secondary sex characteristics are localized, and all other primary and 
secondary sex characteristics are normal. 


Section 12 


Corresponding to the two kinds of pathological variability in 
secondary sex characteristics attributable to tissue under-responsiveness 


and over-responsiveness, respectively, are two parallel kinds of pathological 
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variability attributable to hormonal, and oversecretion of opposite-sex 
hormones. These abnormalities of hormonal origin are obviously much more 
generalized in the number and kinds of organs involved than the tissue 
responsiveness varieties. 

Undersecretion of sex-appropriate gonadal hormones (e.g., androgens 
in males), due either to gonadal pathology or to pathology in the gonadal- 
stimulating protions of the anterior pituitary gland results in 
generalized underdevelopment of all sex-appropriate secondary sex 
characteristics (e.g., iMpueei ricer facial hair, high-pitched voice, 
etc.), as well as underdeveloped primary sex organs. The opposite-sex 
secondary sex characteristics, however, are normally developed. 

Section 13 

On the other hand, oversecretion of the adrenal cortex (due either 
to hypertrophy of the adrenal cortex or to overstimulation of the 
adrenal cortex by the anterior pituitary) results in generalized over- 
development of opposite-sex secondary sex characteristics. In teenage 
girls, for example, excessive secretion of androgens by the adrenal 
cortex stimulates the overgrowth of body and facial hair, gives rise to 
low-pitched voice, etc. But in this instance, primary sex organs and 
sex-appropriate secondary sex characteristics are normally developed. 

Both of these quite different but similarly general pictures of 
pathological pubescence caused by hormonal factors-~gonadal under- 
secretion and adrenal cortex oversecretion--occur extremely rarely. Over- 
or under-responsiveness of just one particular tissue to normal hormonal 
stimulation--with a highly localized type of abnormality--is a much more 


commonly encountered cause of pathological variation. 
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NOTE: This organizer represents the major ideas in the materials you will 
be reading. You might find it useful to examine it thoroughly 
before proceeding to the text. 
The Control of Human Pubescence 

Pubescence is a good example of biological change or development 
occurring during the life cycle of human beings. Like all other forms of 
cyclical biological change in individuals, pubescence is characterized 
both by uniformity (aspects of change that are similar in all human 
beings), and by variability (aspects of change that vary from one person 
to another). Uniformity in pubescence, for example, is illustrated by 
the fact that all persons of a given sex undergo the same kinds of 
pubescence bodily changes in the same orderly, predictable sequence. 
Variability in pubescence is illustrated by the fact that individuals 
differ from each other in the age at which they become pubescent and in 
the degree to which they manifest these same kinds of pubescent traits 
(e.g., size of breasts, amount of pubic hair). This variability tends 
to follow the bell-shaped distribution curve. 

Thus there are some aspects of pubescence (i.e., kind and sequence 
of changes) in which all human beings of a given sex group develop in 
exactly the same way, and other aspects of pubescence (Lieis Agé oF 
puberty, and degree of development) in which a normal range of variability 
exists. Considerable variability both in age of pubescence and in degree 
of development of the various pubescent changes is compatible with normal- 
ity. At the very extremes of the bell-shaped distribution curve, however, 
variability is often pathological in nature. 

The two most important sources of variability in pubescent traits 
are: (a) individual differences in hormonal output, and (b) individual 


differences in the responsiveness of particular tissues (e.g., sex organs, 
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breasts) to hormonal stimulation. Under- or oversecretion of a particular 
hormone results in generalized under- or overdevelopment ofeald sthe 
tissues stimulated by that hormone. Under- or over-responsiveness results 
in a strictly localized under- or overdevelopment of that particular 
tissue. Pathological variations in pubescent characteristics, also tend 
to be either generalized or localized in nature. 

It is also possibie to distinguish between two main types of 
pubescent abnormalities. An individual may be abnormal in the sense that 
he is deficient or underdeveloped with respect to a pubescent trait or 
traits appropriate for his sex group. A pubescent boy, for example, may 
be lacking in bodily and facial hair, and a pubescent girl may have 
atypically small breasts. This type of abnormality reflects undersecretion 
of the sex-appropriate sex hormone or under-responsiveness of the tissue 
in question to this hormone in the affected individual. 

The extent to which variability in a pubescent characteristic is 
either restricted within a given sex group, or exhibits varying degrees 
of overlapping between the sexes, depends on whether the characteristic 


in question is a primary or secondary sex characteristic. This will be 


discussed in the following passage. 
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This organizer represents the major ideas in the materials you will be reading. You might find 


NOTE: 
it useful to examine it thoroughly before proceeding to the text. 


{ 
' 


PUBESCENCE: The biological changes which occur in people during sexual maturation. 


CHARACTERIZED BY: 


I, SAMENESS 
All people of the same sex 
experfence the same KINDS 
of changes in the same ORDER 


i= 


I]. DIFFERENCES 

All people of the same sex differ somewhat 
in the AGE at which the changes begin and 
the DEGREE of change which occurs 
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Content-referenced Objectives 


After reading this passage the student should be able to describe the 
factors which affect the order of pubescent changes in boys and pirls. 


After reading this passage the student should be able to name two 
hormones and their effects on the adrenal cortex and gonads. 


After reading this passage the student should be able to describe the 
differences between the types of variability found in primary and 
secondary sex characteristics. 


After reading this passage the student should be able to describe, 
using examples, the two types of distributions of secondary sex 
characteristics. 


After reading this passage the student should be able to name the 
distribution patterns of primary and secondary sex characteristics. 


After reading this passage the student should be able to describe the 
selective action of sex hormones in terms of primary and secondary 
sex characteristics. | 


After reading this passage the student should be able to describe 
the ways in which hormonal, tissue, hereditary and environmental 
factors influence the age of onset of pubescence. 


After reading this passage the student should be able to describe the 
hormonal and organic causes of pathologically early pubescence. 


After reading this passage the student should be able to describe the 
primary condition reflected by pathologically delayed pubescence and 
its causes. 


After reading this passage the student should be able to describe the 
signs of the two types of pathological variability in primary sex 
characteristics. 


After reading this passage the student should be able to give examples 
for boys and girls of a pathological condition which could be caused 
by tissue enderszesponsiveness to same-sex: hormones. 


After reading this passage the student should be able to describe the 
pathology that leads to underdevelopment of all sex-appropriate 
secondary sex characteristics. 


After reading this passage the student should be able to describe the 
pathology that leads to overdevelopment of opposite-sex secondary 
characteristics and normal primary characteristics. 
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Content-free obiectives 


After reading the next passage of text you should be able to answer 
a multiple-choice type of question dealing with one specific piece 
of information. 


This objective was inserted at the top of each section for which the 
practice question was of the "verbatim" multiple-choice type. 


After reading the next passage of text you should be able to answer 
a multiple-choice type of question dealing with several specific 
pieces of information. 


This objective was inserted at the top of each section for which the 
practice question was of the "paraphrase" multiple-choice type. 


After reading the next passage of text you should be able to answer 
an open-ended type of question dealing with several specific pieces 
of information. . 


This objective was inserted at the top of each section for which the 
practice question was an open-ended type. All open-ended questions 
were "paraphrase". 
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Inconsistent Questions 


What factors do not affect the order of pubescent changes 
in either boys or girls? 


Factors such as age, race, climate and culture do not 
affect the order of pubescent changes in boys or girls. 


What two hormones does the anterior pituitary produce during 
pubescence and what are the effects of these hormones on the 
adrenal cortex and gonads? 


The anterior pituitary produces corticotropic hormones 
which stimulate the adrenal cortex's production of androgen 
and estrogen. The anterior pituitary also produces 
gonodotropic hormones which stimulate the gonads to produce 
same-sex hormones. 


What is the difference in types of variability found in 
primary and secondary sex characteristics? 


Primary sex characteristics can only be over- or under- 
developed while secondary sex characteristics may be over- 
developed in terms of opposite-sex characteristics and 
underdeveloped in terms of same-sex characteristics. 


What are the two types of distribution of secondary sex 
characteristics? Give an example of each. 


Secondary sex characteristics such as axillary hair are 
distributed in an overlapping way while other character- 
istics such as facial hair are more characteristic of one 
sex group than the other and show overlapping only at the 
extremes. 


What are the names of the distribution patterns of primary 
and secondary sex characteristics? 


The pattern of distribution of primary sex characteristics 
occurs in either one sex or the other and is called 
"dichotomous". The pattern of distribution of secondary 
sex characteristics is either equal in both sex groups or 
more pronounced in one group than the other and is called 
"continuous overlapping" or "overlapping at the extremes." 


Describe the selective action of sex hormones in terms of 
primary and secondary sex characteristics. 


Sex hormones have a selectivly stimulating effect on primary 
sex characteristics and sex-appropriate secondary character- 
istics. They have a selectively inhibiting effect on 
opposite-sex primary and secondary sex characteristics. 
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How do hormonal, tissue, hereditary and environmental 
factors determine the age of onset of pubescence? 


The age of onset of pubescence is determined by the age 

at which gonadotropic and gonadal hormones: are secreted 

and the age at which increased tissue responsiveness occurs. 
The age of these events is determined by hereditary and 
environmental factors. 


What are the hormonal and organic causes of pathologically 
early pubescence? 


Pathologically early pubescence might be caused by excessive 
secretions of androgen and estrogen by the adrenal cortex 
because of either hypertrophy of the adrenal cortex itself 
or hypertrophy of the cortex stimulating portion of the 
anterior pituitary. | 


What is the primary condition reflected by pathologically 
delayed pubescence and what are its causes? 


The primary condition reflected by pathologically delayed 
pubescence is gross gonadal insufficiency and might be 
caused by: destructive lesions of the ovaries or testes, 
destructive lesions of the anterior pituitary, prolonged 
or severe malnutrition, or protracted or severe emotional 
trauma. 


What are the signs of the two types of pathological variabil- 
ity in primary sex characteristics? 


Hormonal types of pathological variability are associated 
with generalized underdevelopment of primary sex character- 
istics and sex-appropriate secondary sex characteristics. 
Tissue responsiveness types of variability are associated 
with localized underdevelopment of specific tissues and 
normal development of all other sex characteristics. 


Give an example of one secondary sex condition in boys and 
one secondary sex condition in girls that could be caused 
by localized tissue under-responsiveness to: same-sex hormones. 


Localized tissue under-responsiveness to same-sex hormones 
might lead to lack of facial hair in boys and underdeveloped 
breasts in girls. 


What is the underlying pathology that leads to underdevelop- 
ment of all sex-appropriate secondary characteristics? 


The underlying pathology leading to underdevelopment of 

all sex-appropriate secondary sex characteristics is hormonal, 
due to either pathology of the gonads or the gonal stimulat- 
ing portion of the anterior pituitary. 
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What is the underlying pathology that leads to overdevelop- 


ment of opposite-sex secondary sex characteristics but 
normal primary sex characteristics? 


The appearance of opposite-sex secondary sex characteristics 
but normal primary sex characteristics is caused by over- 
secretion of the adrenal cortex. This oversecretion may 

be caused by pathology of the adrenal cortex or of the 
cortex stimulating portion of the anterior pituitary. 


Consistent Questions 


The characteristic order of pubescent changes in both 
sexes is dependant on: 


a) age 
b) climate 
c) race 


d) culture 
e) none of the above 


The characteristic order of pubescent changes in both 
sexes is dependant on: (e) none of the above. 


During pubescence the anterior pituitary produces which of 
the following pairs of hormones with what effect: 


a) corticotropic and gonadotropic; stimulates gonads and 
adrenal cortex respectively 

b) gonadotropic and gonadal; stimulates gonads and adrenal 
cortex respectively 

c) gonadotropic and corticotropic; stimulates gonads and 
adrenal cortex respectively 

d) corticotropic and sex hormones; stimulates gonads and 
adrenal cortex respectively 

e) sex hormones and corticotropic; stimulates gonads and 
adrenal cortex respectively 


During pubescence the anterior pituitary produces: (c) 
gonadotropic and corticotropic; stimulates gonads and 
adrenal cortex respectively. 


The major group of pubescent changes which occur only within 
a sex group are: ae 


a) primary 

b) secondary 

c) opposite sex 

d) same sex 

e) none of the above 
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Answer 3. The major group of pubescent changes which occur a 
within a sex group are: (a) primary. 


Question 4. Which of the following statements best describes the dis- 
tribution of primary and secondary sex characteristics? 


a) primary characteristics are found in both sexes while 
secondary characteristics are found only in one sex 
group. 

b) primary characteristics are distinctive of one sex 
group while secondary characteristics are found only in 
males or females. 

c) primary characteristics are distributed only within a 
sex group while secondary characteristics are distributed 
between both sexes. 

d) primary and secondary characteristics both show vari- 
ability between and within sex groups. 

e) primary but not secondary characteristics show variabil- 
ity between and within sex groups, 


Answer 4, c) primary characteristics are distributed within a sex 
group while secondary characteristics are distributed 
between both sexes. 


Question 5. Which type of distributions described below are characteris- 
tic of primary or secondary sex groups respectively? 


a) continuous overlapping; overlapping at the extremes and 
dichotomous. 

b) continuous overlapping; dichotomous and overlapping at 
the extremes. 

c) dichotomous; continuous overlapping and overlapping at 
the extremes. 

d) overlapping at the extremes, continuous and dichotomous. 

e) overlapping at the extremes, non-overlapping and con- 


tinuous. 
Answer 5. The characteristics of primary and secondary sex groups 
respectively are: (c) dichotomous; continuous overlapping 


and overlapping at the extremes. 


Question 6. The male sex hormone androgen selectively stimulates the 
growth of: 


a) testes and facial hair and inhibits the growth of 
axillary hair. 

b) testes and inhibits growth of breasts. 

c) axillary hair and inhibits growth of breasts. 

d) facial and axillary hair and inhibits the growth of the 


uterus. 
e) breasts and facial hair and inhibits the growth of the 


uterus. 


The male sex hormone androgen selectively stimulates the growth 


Answer 6. 
of: (b) testes and inhibits the growth of breasts. 


ae 
ee a) | 
Mast aed dtm Se : ce . mS a a 
‘ wit, i ae ea ) "aah i, A rey : in 
ote : f Pisa ca rt mh ps te708 woh am 


x52: 460) doe 
ni wilde bowed ars 


s atdstw Sage beau ai ab 


batads Tole b 5 aeibsinh tas ‘ 


fl * Fae 

ae oe 2B 
oe yhy wit Atod Aoi 38 
elidéi tev ‘dart conte 


x96 eo atdaiy bedadis3gF8) 6 P 
peyudiadalb exs eotretredde 


~eiisibssert: 52a woted bod otsugtyiesh 2 
ty lad obese equotdenge Antebaodie’ 4d! . 
hed ase twats. si ihe entuceivave  ntocatane 8 


& gabeen.s vr fas has 


weoatod Gita NO bids OU NGALINOD p ec , a oe 
“<f1090 bus Sate, icine we ; ( 


equionig’ xoa etabicese bing Gina 
antagsixave 2oogadtApas aig A alk 


Sia eend Se" oo 
-etehsad Jo if rE 
$2 10.div6rxR sald estetiind * 


Ciworg sda cadeluaide yisvis oeles 
‘vJeagae To djworg evict 


95 


Question 7. The average age of normal pubescence will be earlier in 
conditions of: 


a) extreme climates such as tropical or arctic. 
b) prolonged moderate malnutrition. 

c) good nutrition and temperate climate. 

d) good nutrition and tropical climates. 

e) good nutrition and arctic climates. 


Answer 7. The average age of normal pubescence wilt be earlier’ in 
conditions of: (c) good nutrition and temperate climate. 


Question 8. Precocious puberty is almost always caused by excessive 
secretion of: 


a) the adrenal cortex sometimes caused by overstimulation 
of the adrenal cortex by the anterior pituitary. 

b) the gonads sometimes caused by overstimulation of the 
gonads by the adrenal cortext. 

c) the anterior pituitary sometimes caused by overstimulation 
of the pituitary by the adrenal cortex. 

d) the adrenal cortex sometimes caused by overstimulation of 
the cortex by the gonads. 

e) the gonads sometimes caused by overstimulation of the 
gonads by the anterior pituitary. 


Answer 8. Precocious puberty is almost always caused by excessive 
secretion of: (a) the adrenal cortex caused by overstimu- 
lation of the cortex by the anterior pituitary. 


Question 9. Pathologically delayed pubescence reflecting extreme insuffi- 
ciency of gonadal production might be caused by: 


a) damaged or malfunctioning ovaries or testes. 

b) destructive conditions of the anterior pituitary. 
c) destructive conditions of the adrenal cortex. 

d) both a) and b) above. 

e) both a) and c) above. 


Answer 9. Pathologically delayed pubescence reflecting extreme 
insufficiency of gonadal production might be caused by: 
d) damaged or malfunctioning ovaries or testes and destruc- 
tive conditions of the adrenal cortex. 


Question 10. Underdevelopment of primary sex characteristics which are 
hormonally caused lead to: 


a) localized underdevelopment of a specific tissue. 

b) pathologically delayed pubescence. 

c) underdevelopment of primary and sex-appropriate secondary 
sex characteristics. 

d) both a) and b) above. 

e) both b) and c) above. 
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Underdevelopment of primary sex characteristics which are 
hormonally caused lead to: e) pathologically delayed 
pubescence and underdevelopment of primary and sex-appro- 
priate secondary sex characteristics. 


Localized tissue under-responsiveness to same-sex hormones 
in boys and girls respectively might lead to: 


a) high-pitched voice and underdeveloped breasts. 

b) lack of axillary hair and overdeveloped facial hair. 

c) lack of facial hair and overdeveloped breasts. 

d) lack of facial hair and development of miniature beard. 
e) development of breasts and miniature beard. 


Localized tissue under-responsiveness to same-sex hormones in 
boys and girls respectively might lead to: (a) high- 
pitched voice and underdeveloped breasts. 


The secondary sex characteristics abnormalities produced by 
over-responsiveness to opposite-sex hormones and under-respon- 
siveness to same-sex hormones are: 


a) generalized and involve several sex characteristics. 
b) localized and involves all sex-appropriate characteris- 


tics. 

c) generalized and involve only opposite sex secondary 
characteristics. 

d) localized and involve only one secondary sex character- 
istic. 


e) generalized and all other secondary sex characteristics 
will be normal. 


The secondary sex characteristic abnormalities produced by 
over-responsiveness to opposite-sex hormones and under-respon- 
siveness to same-sex hormones are: (d) localized and 

involve only one secondary sex characteristic. 


Abnormal development of opposite-sex secondary characteris- 
tics but normal primary characteristics may indicate under- 


lying pathology in: 


a) the cortex stimulating portion of the anterior pituitary. 
b) the gonad stimulating portion of the anterior pituitary. 
c) the adrenal cortex. 

d) both a) and c) above. 

e) both b) and c) above. 


Abnormal development of opposite-sex characteristics but 
normal primary characteristics may indicate underlying 
pathology in: (d) both a) the cortex stimulating portion 
of the anterior pituitary and the adrenal cortex, and c) 
the adrenal cortex. 
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Pubescence is initiated by the which increases pro- 
duction of and begins production of : 


a) 
b) 
c) 
d) 
e) 


adrenal cortex, corticotropic hormones, gonadotropic hormone. 
anterior pituitary, gonadotropic, sex hormones. 

anterior cortex, sex hormones, gonadotropic hormones. 

anterior pituitary, corticotropic hormones, gonadotropic hormones. 
adrenal cortex, gonadotropic hormones, same-sex hormones. 


Precocious puberty is almost always caused by: 


a) 
b) 
c) 
d) 
e) 


overproduction from the adrenal cortex. 
overproduction from the gonads. 
understimulation of the gonads. 
understimulation of the adrenal cortex. 
understimulation of the anterior pituitary. 


Primary sex characteristics are: 


a) 
b) 
c) 
d) 


e) 


by definition confined to only one sex group and can only vary 
between sexes. 

the primary organs of reproduction and vary only within the sex 
groups. 

all those sex-specific characteristics and can therefore vary only 
within the sex groups. 

variable only in terms of over- and underdevelopment and therefore 
relevant only with a sex group. 

variable only in terms of underdevelopment between sex groups. 


The bodily changes of pubescence do not begin spontaneously but rather 
tend to follow each other: 


a) 
b) 
) 


d) 
e) 


in a definite order which is the same for all persons in a given 
sex group. 

in a random order which varies among people in different climates 
and cultures. 

in a definite order which is different for people of different 
age groups. 

in a random order which is the same for people of the same race. 
in a definite order which varies only as a product of extreme 


malnutrition. 


Secondary sex characteristics may be: 


a) 
b) 
c) 
d) 
e) 


dichotomously distributed. 

overlapping at the extremes. 

completely overlapping. 

both dichotomously distributed and «ont tee overlapping. 
both overlapping at the extremes and completely overlapping. 


Select the phrase which best describes the effects of sex hormones: 


a) 


male and female hormones balance each other chemically. 
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b) male and female sex hormones have either a selective or unselec- 
tive effect on sex characteristics. 

c) male and female sex hormones unselectively stimulate their sex 
appropriate characteristics. 

d) male and female hormones selectively inhibit some primary 
characteristics. 

e) male and female hormones selectively stimulate only sex appropri- 
ate secondary sex characteristics. 


Before and after pubescence the adrenal cortex produces what two 
hormones in response to what other hormone? 


a) androgen and gonadal; corticotropic. 
b) androgen and estrogen; gonadotropic. 
c) estrogen and gonadal; corticotropic. 
d) corticotropic and gonadotropic; estrogen. 
e) estrogen and androgen; corticotropic. 


Which of the following examples represents a Seat ekg Re ha which is 
"overlapping at the extremes''? 


a) development of rudimentary breasts in males. 
b) development of some facial hair in females. 
c) development of more facial hair by some females than some males. 


‘d) similar development of axillary and pubic hair among both males 


and females. 
e) development of more axillary hair by some females than some males. 


Variability in primary sex characteristics is characterized by: 


a) underdevelopment only within a sex group. 

b) over- and underdevelopment within a sex group. 
c) underdevelopment only between sex groups. 

d) overdevelopment only among both sexes. 

e) overdevelopment only between sex groups. 


i) Factors which determine individual differences and ii) the 
average ages of pubescence in males and females are: 


a) hormonal; 13.5 and 14.0 years. 

b) tissue responsiveness; 14.5 and 13.0 years. 

c) hormonal and tissue responsiveness; 14.5 and 13.0 years. 
d) hereditary and environmental; 14.5 and 13.0 years. 

e) hereditary and hormonal; 14.5 and 13.0 years. 


Factors which cause pubescence to occur outside the average range of 
10 to 17 years are: 


a) hereditary and environmental. 

b) hormonal. 

c) environmental and hormonal. 

d) hormonal and tissue responsiveness. 
e) related to tissue responsiveness. 
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The invariant order of pubescent changes is best explained in terms 
of: 


a) differential hormonal effects. 

b) selective and unselective effects of hormones. 

c) differential tissue responsiveness to hormonal stimulation. 
d) umselective tissue responsiveness to hormonal stimulation. 
e) unequal concentrations of hormones in selective tissues. 


The term "dichotomous non-overlapping" distribution is accurate only 
in terms of: . 


a) secondary sex characteristics which are differentiated with 
respect to sex groups. 

b) primary sex characteristics which are more specific to one sex 
group than another. 

c) secondary sex characteristics which are undifferentiated with 
respect to sex groups. 

d) both primary and secondary sex characteristics which are 
differentiated with respect to sex groups. 

e) primary sex characteristics which exist only within sex groups. 


The kinds and sequences of pubescent changes in precocious puberty 
are: 


a) very similar to normal puberty but secondary characteristics 
more marked. 

b) very similar to normal puberty but primary characteristics more 
marked. 

c) identical in every respect to those occurring in normal puberty. 

d) identical with the exception of slight variations in order of 
changes. 

e) identical with the exception of slight variations in kind of 


changes. 
The first pubescent changes in (i) boys and (ii) girls are: 


a) changing voice, pubic hair. 

b) pubic hair, enlargement of breasts. 

c) growth of testes, first menstruation. 

d) growth of testes, enlargement of breasts. 
e) increased strength, enlargement of breasts. 


Variability in secondary sex characteristics is characterized by: 


a) both normal and pathological variability within a sex group. 
b) underdevelopment of sex appropriate characteristics. 

c) overdevelopment of opposite sex characteristics. 

d) both pathological and normal variability between sex groups. 
e) only over- or underdevelopment of same-sex characteristics. 
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Which of the following statements describes most of the conditions 
which might produce abnormally delayed pubescence: 


a) 
b) 
c) 
d) 


e) 


destructive lesions of the gonads, malnutrition, emotional 
trauma. 

destructive lesions of the adrenal cortex, severe malnutrition, 
prolonged emotional trauma. 

destructive lesions of the gonads or anterior pituitary, severe 
malnutrition, prolonged emotional trauma. 


destructive lesions of the gonads and adrenal cortex, malnutrition, 


and severe emotional trauma. 


destructive lesions of the anterior pituitary, severe malnutrition, 


prolonged or severe emotional trauma. 


The only type of variability in primary sex characteristics of 
medical and behavioral significance is: 


a) 
b) 
c) 
d) 
e) 


hermaphrodism. 

cryptochidism. 
overdevelopment. 

opposite sex characteristics. 
underdevelopment. 


Generalized underdevelopment of same-sex secondary sex characteris-— 
tics in a person with normal primary sex characteristics suggests: 


a) 
b) 
c) 
d) 
e) 


high adrenal cortext output. 
low gonadotropic output. 

low gonadal output. 

high gonadotropic output. 
high gonadal output. 


Localized tissue under-responsiveness: 


a) 
b) 


c) 
d) 
e) 


causes sex appropriate mixup in males. 

causes underdevelopment of certain sex appropriate sex chacter- 
istics: 

causes overdevelopment of certain opposite sex characteristics. 
causes delayed puberty. 

causes precocious puberty. 


Which of the following treatments are appropriate for abnormally 
delayed pubescence resulting from lesions of the ovaries or testes: 


a) 
b) 
c) 
d) 
e) 


medical or surgical treatment of the anterior pituitary. 
medical or surgical treatment of the adrenal cortex. 
injections of opposite-sex hormones. 

injections of sex appropriate hormones. 

both a) and d) above. 


Localized abnormalities such as overdeveloped breasts in males is 
most commonly caused by: 
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a) gross gonadal insufficiency. 

b) severe and protracted emotional trauma. 

c) localized tissue under-responsiveness. 

d) underdeveloped opposite-sex characteristics. 
e) low gonadotropic output. 


The most common cause of underdeveloped primary sex characteristics 
is: 


a) pathologically delayed pubescence. 

b) localized tissue under-responsiveness. 
c) generalized tissue under-responsiveness. 
d) gonadal insufficiency. 

e) generalized hormonal inadequacy. 


An abnormality which involves one secondary sex characteristic and 
leaves all other secondary and primary characteristics normal is 
probably produced by: 


a) gross gonadal insufficiency. 

b) severe or protracted malnutrition or emotional trauma. 

c) localized tissue over- or under-responsiveness. 

d) generalized hormonal insufficiency. 

e) localized tissue under-responsiveness to opposite sex hormones. 


Generalized underdevelopment of sex appropriate secondary sex 
characteristics might develop from: 


a) localized hormonal insufficiency. 

b) localized tissue under-responsiveness. 

c) generalized tissue under-responsiveness. 

d) generalized insufficiency of the adrenal cortex. 
e) generalized insufficiency of the gonads. 


Generalized overdevelopment of opposite-sex secondary sex character- 
istics might occur from: 


a) generalized gonadal overproduction. 

b) localized tissue over-responsiveness. 

c) generalized tissue over-responsiveness. 

d) localized overproduction of gonadotropic hormone. 
e) generalized oversecretion by adrenal cortex. 
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APPENDIX VII 
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Questionnaire*x 


This research involves ways in which we can improve learning from text. 
Three different types of instructions were involved (organizers, 
objectives, and questions). I would appreciate your answers to the 
following questions. 


es 


How useful did you find the materials you were given to study? 


extremely somewhat not very not useful 
useful useful useful useful at all 


be inl ace Sal Lieierall [iid | Lee) 


Did you try to use the organizers, objectives, and questions as you 
studied? 


used very didn't use 
used all used most used a few few any 


eed ay Usp lennae' a a aR se a 


How interesting did you find learning from this material? 


very somewhat not very not interest- 
interest interesting interesting interesting ing at all 


Pall lapess f ed ‘Sean eas 


How do you prefer to learn from text materials? (Select as many 
answers as are appropriate to you) 


a) just by reading e) by reciting things to myself 
b) by reading several times f) by answering questions 
c) by making notes g) other (describe) 


d) by underlining 


Please make any comments or ask any questions you wish regarding this 
research. 


For all except "text only" control group. 
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Questionnaire* 


This research involves ways in which we can improve learning from text. 
I would appreciate your answers to the following questions. 


1. How useful did you find the materials you were given to study? 


extremely somewhat not very not useful 
useful useful useful useful at all 


Bee tly tS [ee StS eee ee 


2. How interesting did you find learning from this material? 


very somewhat not very not interest- 
interesting interesting interesting interesting ing at adc 


Ol AN ry Fata) eae Fa Pee aes 


3. Which of the following techniques or strategies did you use in 
studying this material? 


a) reading only 
b) reading and rereading 

c) making notes 

d) underlining 

e) reciting things silently 
f) asking myself questions 
g) other (describe) 


So nn ee EEE UIEnnEEnnE EEE 


4. Please make comments or ask any questions you wish. 


* 
For "text only" control group. 
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APPENDIX VIII 


Introduction and General Instructions 
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Introduction 
The following general information was given by the experimenter to 


each group before they began to study: 


I am a graduate student working in Educational Psychology and my 
particular interest is Instructional Psychology or Psycho-educational 
Design. This means that I am involved in an attempt to draw together 
information from various areas of research and theory to help provide 
ideas and information which will be useful in applied situations. 

I have decided to work with text or prose materials because we still 
use text books and papers for a great deal of our learning experiences. 

The potential benefits to you in helping me with this research is 
marginal for you personally. The real benefits possible are to you and 
I both as people who want to help promote learning. 

As teachers you will often call upon learning resources prepared by 
other people for use with your students. I am one of these "other people 
who is actively engaged in preparing instructional materials. 

In order for me to prepare effective instructional materials it is 
important for me to investigate some of the variables involved in their 
effectiveness. Hopefully, some of the information I obtain, somewhere 
along the line in your teaching careers, will come back to you in a form 
that will help you help your students. 

This study will take approximately one hour of your time. Your 
professor has permitted me to conduct this study during your regular 
classroom time. You will be given a package of materials, time to study, 


and then will be asked to write a test. The scores obtained on the test 


will be annonymous and will not be reflected in your course grades. 
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General Instructions 
This package contains some text materials for you to study. Please 
use whatever techniques or strategies you usually use in studying from 
this kind of material. After 35 minutes you will be given a test. 


(text only control group) 


This package contains some text materials for you to study. Please 
follow any instructions you find within the package. After 35 minutes 


studying you will be given a test. (all groups except text-only control) 


Achievement Test Instructions 
Please read each question carefully and select ONE answer from the 
alternatives given. Answer each question. Please do not leave any 


blanks. 


Delayed Test Introduction 


You were not told that I would be returning to ask you to write 
another test for two reasons. First, some people become nervous and worried 
when they know they will have to write a test. I did not want you to 
worry about this test. Second, some people, when they know they will be 
tested try to prepare themselves, for example, by finding out answers to 
questions they did not know or by studying from other acer aie I am 
interested in finding out about what you learned when you studied the 
materials I gave you. Therefore, I did not want anyone to do any outside 
studying. 


You will have 20 minutes to write this test. Please work as 


quickly and carefully as you can. 
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